Complete 3D Face Recovery: Hybrid Techniques for Occlusion and Pose- Invariant Biometric Recognition

Abstract Robust facial analysis is a critical challenge for biometric systems, particularly in the presence of pose variation and significant occlusion. This paper introduces a self-supervised framework, Complete Face Recovery GAN (CFR-GAN), which leverages 3D Morphable Models and a Swap-Rotate-and-Render (Swap-R&R) strategy to synthesise natural frontal, de-occluded facial images from single, occluded, or non-frontal inputs. The model combines a U-Net-based generator with an occlusion-aware parsing branch and a multi-scale discriminator, all trained with a composite loss that comprises adversarial, identity, perceptual, and mask terms. Empirical evaluations on CelebA-HQ, FFHQ, and Multi-PIE datasets confirm the superiority of this approach over existing methods, both in terms of recognition accuracy and qualitative restoration, eliminating the need for paired or curated training images.