Advancing Emotion Detection in Affective Computing: Exploring Physiological Signal Responses

Affective computing through the monitoring of physiological signals is a prominent topic in both scientific research and industry. Numerous wearable devices are being developed for health and wellness tracking during daily activities and sports. Similarly, applications are emerging for the early detection of risk situations, such as sexual or violent assaults, by identifying emotions like fear or panic. Incorporating additional data sources, such as video or audio signals, can enhance the capabilities of multimodal affective computing, making emotion classification more accurate. Other biological factors, which have yet to be thoroughly explored, could also contribute to better differentiation of negative emotions like fear or panic. Catecholamines, hormones produced by the adrenal glands above the kidneys, are released in response to physical or emotional stress. The primary catecholamines—adrenaline, noradrenaline, and dopamine—have been analyzed alongside four physiological variables: skin temperature, electrodermal activity, blood volume pulse (used to calculate heart rate), and respiration rate. This study compares the results of physiological signal analysis and catecholamine levels in an experiment involving 21 female participants exposed to audiovisual stimuli in an immersive virtual reality environment. AI algorithms have been used to classify fear based on physiological variables and plasma catecholamine concentrations. The best results were achieved using features derived from the physiological signals. However, adding the maximum variation of catecholamine levels during the five minutes following the video or including five interval measurements (one per minute) did not improve classifier performance.