Large Margin Nearest Neighbor Classification With Privileged Information for Biometric Applications

In this paper, a new metric learning algorithm is proposed to improve face verification and person re-identification in RGB images by learning from RGB and Depth (RGB-D) training images. We address this problem by formulating it as a Learning Using Privileged Information problem, in which the additional depth images associated with the RGB training images are not available for the testing process. Based on the large margin nearest neighbor (LMNN) classification framework, we propose an effective metric learning method called large margin nearest neighbor classification with privileged information (LMNN+) by incorporating depth information to improve decision function learning in the training process. Specifically, two distance metrics based on visual features as well as depth features are jointly learned by minimizing the triplet loss in which the within-class difference is minimized, while the between-class difference is maximized. The distances in the depth feature space can be utilized to guide the training process in the visual feature space. In addition, we propose an efficient optimization method which can handle billions of constraints in the optimization problem of LMNN+. The comprehensive experiments on the EUROCOM data set, the CurtinFaces data set as well as the BIWI RGBD-ID data set demonstrate the effectiveness of our algorithm for face verification and person re-identification by leveraging privileged information.