Hallucinating the Full Face from the Periocular Region via Dimensionally Weighted K-SVD

Identifying a suspect wearing a mask (where only the suspect's periocular region is visible) is one of the toughest real-world challenges in biometrics that exist. In this paper, we present a practical method to hallucinate the full frontal face given only the periocular region of a face. This is an important problem faced in many law-enforcement applications on almost a daily basis. In such real-world situations, we only have access to the periocular region of a person's face. Unfortunately commercial matchers are unable to process these images successfully. We propose in this paper, an approach that will reconstruct the entire frontal face using just the periocular region. We empirically show that our reconstruction technique, based on a modified sparsifying dictionary learning algorithm, can effectively reconstruct faces which we show are actually very similar to the original ground-truth faces. Further, our method is open set, thus can reconstruct any face not seen in training. We show the real-world applicability of method by benchmarking face verification results using the reconstructed faces to show that they still match competitively compared to the original faces when evaluated under a large-scale face verification protocol such as NIST's FRGC protocol where over 256 million face matches are made.