Feature-Level Fusion of FaceNet and Wav2vec Representations for Multimodal Biometric Authentication

Biometrics are highly popular due to their ease and convenience in utilizing unique human physical characteristics.Biometrics has advanced, and multimodal biometrics are more accurate than unimodal biometrics.Multimodal biometrics involves the combination of two or more biometric modalities.This study investigates multimodal biometric authentication using face and voice modalities with deep learning models.The FaceNet and wav2vec models are utilized for feature extraction from face images and voice samples of 50 subjects from the MSU-AVIS dataset.Feature level and score level fusion strategies are implemented.Using the Euclidean distance and classifiers methods such as Decision Tree, Support Vector Machine (SVM), Random Forest, XGBoost, and Artificial Neural Network (ANN), performance is evaluated by correct acceptance rate (CAR) at 0.1 and 0.01 false acceptance rates and area under the ROC curve (AUC).The proposed feature level fusion approach with XGBoost obtained the highest CAR of 0.995 at 0.01 FAR and AUC of 0.996 on the test set, demonstrating effective multimodal biometric authentication.