Convolutional Experts Constrained Local Model for 3D Facial Landmark Detection

Constrained Local Models (CLMs) are a well-established family of methods for facial landmark detection. CE-CLM, the newest member of CLMs, brings CLMs back to state of the art performance. This is done through CE-CLMs ability to model the very complex individual landmark appearance that is affected by expression, illumination, facial hair, makeup, and accessories. A crucial component of CE-CLM is a novel local detector - Convolutional Experts Network (CEN) - that brings together the advantages of neural architectures and mixtures of experts in an end-to-end framework. In this paper we use CE-CLM to learn position of dense 84 landmark positions. To achieve best performance on the Menpo3D dense landmark detection challenge, we use two complementary networks alongside CE-CLM: a network that maps the output of CE-CLM to 84 landmarks called Adjustment Network, and a Deep Residual Network called Correction Networks that learns dataset specific corrections for CE-CLM.