Enhancing Stress Detection Accuracy through Multimodal Fusion by Exploiting Biological Sensors and Audio Signals

Accurate stress detection is vital for various applications, including health monitoring and well-being assessment. This research study addresses the challenge of stress detection through a multimodal fusion approach, leveraging data from biological sensors and audio signals. The biological data recorded by the sensor is preprocessed, and techniques like feature scaling and categorical variable encoding are employed. For the audio data, features such as Zero Crossing Rate (ZCR) and Fast Fourier Transform (FFT) are extracted to capture emotional cues indicative of stress. Several machine learning algorithms are trained and compared on the respective datasets, including Artificial Neural Networks (ANN), Random Forest, Gradient Boosting Machine (GBM), and XG-Boost. The analysis reveals the superior performance and accuracy of GBM (97%) and XG-Boost (94%) in stress detection compared to ANN and Random Forest, highlighting the efficacy of ensemble learning and boosting techniques. The fusion of sensor and audio data enables a comprehensive understanding of stress states, enhancing detection accuracy and robustness. This research work contributes to advancing stress detection systems with potential applications in personal health monitoring, mental health support, and stress management interventions.