Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis

Semantic image synthesis aims at generating photorealistic images from layouts. Previous approaches with conditional generative adversarial (GAN) show state-of-the-art performance on this task, which either the semantic label maps as inputs to the generator, or use them to the activations in normalization layers via affine transformations. We that convolutional kernels in the generator should be aware of the semantic labels at different locations when generating images. In to better exploit the semantic layout for the image generator, we propose predict convolutional kernels conditioned on the semantic label map to the intermediate feature maps from the noise maps and eventually the images. Moreover, we propose a feature pyramid semantics-embedding, which is more effective in enhancing fine details and semantic between the generated images and the input semantic layouts than multi-scale discriminators. We achieve state-of-the-art results on quantitative metrics and subjective evaluation on various semantic datasets, demonstrating the effectiveness of our approach.