Style Adaptation Based on Image Processing Methods Using Cyclegan

Cycle-Consistent Generative Adversarial Networks (CycleGANs) are able to provide a highly under-constrained mapping between input and output data samples, i.e., source and target data domain, in cases when the aligned dataset is unavailable, in an unsupervised training fashion, using cycle-consistency loss mechanisms. On the other hand, most image-to-image and speech-to-speech translation tasks use the aligned, i.e., paired input-output training datasets. A large amount of data is necessary to train such architectures, while one of the domains could be scarce. Several possible improvements to the original CycleGAN architecture are analysed in this paper for the cases when only a small percentage of training samples are aligned among source and target data domains. A semi-supervised approach is proposed to achieve better translation accuracy and prevent overfitting of the scarce data domain discriminator during initial training iterations. The training database is augmented by adding samples generated by inverse CycleGAN mappings after several training epochs (when the network is sufficiently trained) into the training pool of the discriminator of scarce, i.e., reduced data domain. An additional optimization constraint is also proposed, aligning probability distributions of feature maps belonging to the same-depth neural network layers of direct GAN encoder and inverse GAN decoder, to reinforce resemblance among object representations in various data domains. Significantly better performances are obtained using proposed improvements in both image-to-image and speech-to-speech translation tasks, by observing standard qualitative and quantitative measures, in comparison to the baseline CycleGAN training approach.