A secure and explainable multimodal biometric system using trust adaptive fusion for face and fingerprint

Multimodal biometrics are able to improve the accuracy and security of authentication by integrating more than one biometric characteristic, minimizing errors, and maximizing the resistance to attacks. The primary drawback of multimodal biometric verification is the complexity of the systems that are introduced by multiple sensors, more computing, and fusion issues. Multimodal feature extraction methods are inadequate in traditional feature extraction methods as they generate modality-specific, handcrafted representations which are not robust, compatible and discriminative enough to support effective feature-level fusion. Deep learning feature extractors produce robust, discriminative, and fusion-friendly representations which are very important in multimodal biometric authentication systems to enhance accuracy and reliability. Trust and confidence are crucial in multimodal biometric authentication systems utilizing deep learning, as the models operate as black boxes, handle irreversible biometric data, and make high-impact security decisions. This motivates the development of a secure, explainable, multimodal biometric authentication framework. The proposed system is a privacy-preserving and explainable multimodal biometric solution that combines deep learning, trust-adaptive fusion, and encrypted domain matching. It utilizes MobileNet for extracting discriminative features. A Trust Adaptive Fusion (TAF) Strategy adjusts the contribution of each modality based on its quality or confidence, enhancing the robustness against the noisy inputs. The fused features are secured using the Cheon-Kim-Kim-Song (CKKS) homomorphic encryption, without revealing the raw biometric data. Transparency is enhanced with the help of the Grad-CAM, which provides interpretability of the model’s decision. The proposed system is evaluated on the CASIA-FaceV5 and CASIA-FingerprintV5 datasets, demonstrates the low error rate of 0.0038 on fused feature representation.