Dynamic Confidence Stability Modelling Using Temporal Micro-Expression and Vocal Tremor Fusion for AI-Based Interview Assessment

The automated assessment of human psychological states, particularly confidence, is a domain with increasing relevance in artificial intelligence (AI)- driven analytics, including applications such as interview evaluation and performance monitoring. This document presents a novel approach for dynamic confidence stability modelling, integrating temporal micro-expression analysis and vocal tremor fusion. Traditional methods often rely on single modalities or static feature sets, which can limit the nuanced understanding of rapidly fluctuating internal states. Our methodology leverages the subtle, involuntary cues present in both facial micro-expressions and speech patterns, which are known to be indicative of emotional arousal and cognitive load. A multi-stage processing pipeline extracts granular temporal features from video and audio streams. Specifically, facial Action Units (AUs) are analysed for transient, low-intensity movements, while vocal features such as fundamental frequency perturbation (jitter) and amplitude perturbation (shimmer) quantify speech instability. These heterogeneous features are then subjected to a dynamic fusion mechanism, employing a recurrent neural network architecture with attention mechanisms to model their temporal evolution and interdependencies. The resulting fused representation enables the continuous tracking and prediction of confidence levels, yielding a confidence stability curve over time. Evaluation on a bespoke dataset of simulated interviews demonstrates that this multimodal, temporal fusion framework surpasses unimodal baselines and static fusion techniques in accuracy and robustness. The system offers enhanced interpretability by quantifying the contribution of each modality to the overall confidence prediction. This research contributes to more sophisticated, real-time AI analytics for sensitive human interactions, paving the way for adaptive feedback systems and improved human-computer interaction paradigms.