EvalNet: Sentiment Analysis and Multimodal Data Fusion for Recruitment Interview Processing

Abstract Large language models (LLMs) have made strides in various tasks, yet challenges remain in effectively analyzing complex data from multiple sources, particularly in recruitment scenarios. Sentiment analysis in interviews requires comprehension of diverse cues conveyed through audio, visual, and textual modalities. To address this, we introduce EvalNet, a framework designed specifically for recruitment interview processing, enabling a holistic assessment of candidate responses. EvalNet leverages advanced deep learning methodologies to extract features and categorize sentiment in real-time. By integrating multimodal data, EvalNet enhances predictive accuracy significantly compared to traditional methods focused on single modalities. Experiments carried out with a range of recorded interviews illustrate EvalNet's superior performance in sentiment detection and understanding intricate candidate expressions, serving as a valuable tool for recruiters to make evidence-based decisions in recruitment practices.