Multimodal Recognition of Out-of-Distribution Individuals Using Contrastive Learning

This paper presents an innovative methodology detecting out-of-distribution individuals based on a multimodal contrastive learning approach. The system combines voice and facial image data by projecting them into a shared representation in the embedding space, enable accurate identification of previously unseen individuals. This approach overcomes the limitations of traditional methods by providing more robust and consistent detection in dynamic scenarios, using advanced neural networks and optimized contrastive losses. Specifically, the main contribution of this work is the introduction of a multimodal contrastive framework that performs cross-modal consistency verification between facial and vocal representations, enabling reliable detection of out-of-distribution individuals without the need for identity gallery retrieval. Experimental results on multiple datasets highlight the effectiveness of the system, with accuracy above 90% in detecting in-distribution samples in all evaluated cases. Regarding the identification of out-of-distribution cases, the system maintains outstanding performance, achieving values close to 90% on average, with some datasets exceeding 95%. These results underscore its ability to recognize both known identities and handle unknown data, even under challenging conditions. This approach represents a significant advancement in the multimodal recognition of individuals, with potential applications in critical areas such as security, surveillance, and human–computer interaction.