A multimodal transformer based generative adversarial network for age-aware sketch-to-photo synthesis

The synthesis of photorealistic facial images from sparse hand-drawn sketches, while preserving demographic attributes such as age, remains challenging in computer vision. This work presents a multi-task Generative Adversarial Network (GAN) that jointly addresses sketch-to-image translation and age prediction within a unified end-to-end architecture. The generator of the GAN model makes use of a pretrained ResNet-based content encoder with a 4-block Transformer bottleneck followed by a synthesis decoder network. The model utilizes this setup to comprehend dependencies that occur over a significant spatial distance. To align its demographic characteristics with the synthesis process, an age encoder network predicts a normalized age scalar directly from the sketch. The synthesis decoder is dynamically guided by age conditioning with a scheduled sampling curriculum through adaptive instance normalization (AdaIN) to mitigate the discrepancy between training and inference. To maintain consistent age and texture at the patch level, the proposed multi-scale CNN discriminator examines 4-channel inputs (the RGB image plus a uniform age map) at distinct resolutions. The model was trained and tested on different publicly available datasets (IMDB-WIKI, UTKFace and Hand-Drawn CUFS Sketches) and obtained a learned perceptual image patch similarity of 0.1982 with a PSNR of around 20.31 dB. The MAE for age prediction is 9.29 years on unconstrained images and 2.09 years on aligned faces. The architecture is computationally efficient, which makes it an effective baseline for forensics and identity-preserving generation.