The paper offers a technique called CycleGAN, which claims to achieve image-to-image translation without paired data sets,whereas in classical methods like pix2pix, a paired training dataset is required, ie. winter and summer pictures of thousands of landscapes, and provided such dataset, with pix2pix a model can be trained to translate a given test dataset, say summertime landscapes, to wintertime versions. It can be used for style transfer, object transfiguration, season transfer and photograph generations from paintings.
How does it work?
Where CycleGAN differs is that it takes advantage of an additional loss function called Cycle Consistency Loss. The aim is to achieve a minimum error between the input and Back-translation of the translation of the given input.
- Take a Monet painting M and generate a picasso painting F(M) = P
- Generate a Monet painting again, G(P) = M’
- Calculate the error between M and M’ and update the model
And do the same process for Picasso domain and Monet target.
This algorithm describes Cycle Consistency loss, and we of course still need models for mentioned mappings F and G, and also techniques for computing errors. But Cycle Consistency Loss is where CycleGAN differs from other image translation methods.