Speech-based deep learning models successfully predicted physiological markers of stress, yielding a Spearman correlation coefficient of 0.770 for cortisol measurements taken 10 minutes after a stress period.
Observational (n=134)
Yes
Can speech-based features processed by machine learning predict physiological markers of stress in subjects undergoing a stress test?
Speech-based features processed by deep learning can effectively predict physiological markers of stress such as cortisol levels and continuous heart rate.
Effect estimate: Spearman ρ 0.770
Life in modern societies is fast-paced and full of stress-inducing demands. The development of stress monitoring methods is a growing area of research due to the personal and economic advantages that timely detection provides. Studies have shown that speech-based features can be utilised to robustly predict several physiological markers of stress, including emotional state, continuous heart rate, and the stress hormone, cortisol. In this contribution, we extend previous works by the authors, utilising three German language corpora including more than 100 subjects undergoing a Trier Social Stress Test protocol. We present cross-corpus and transfer learning results which explore the efficacy of the speech signal to predict three physiological markers of stress—sequentially measured saliva-based cortisol, continuous heart rate as beats per minute (BPM), and continuous respiration. For this, we extract several features from audio as well as video and apply various machine learning architectures, including a temporal context-based Long Short-Term Memory Recurrent Neural Network (LSTM-RNN). For the task of predicting cortisol levels from speech, deep learning improves on results obtained by conventional support vector regression—yielding a Spearman correlation coefficient ( ρ ) of 0.770 and 0.698 for cortisol measurements taken 10 and 20 min after the stress period for the two corpora applicable—showing that audio features alone are sufficient for predicting cortisol, with audiovisual fusion to an extent improving such results. We also obtain a Root Mean Square Error (RMSE) of 38 and 22 BPM for continuous heart rate prediction on the two corpora where this information is available, and a normalised RMSE (NRMSE) of 0.120 for respiration prediction (−10: 10). Both of these continuous physiological signals show to be highly effective markers of stress (based on cortisol grouping analysis), both when available as ground truth and when predicted using speech. This contribution opens up new avenues for future exploration of these signals as proxies for stress in naturalistic settings.
Baird et al. (Mon,) conducted a observational in Stress (n=134). Speech-based machine learning models (LSTM-RNN) vs. Conventional support vector regression (SVR) was evaluated on Prediction of sequentially measured saliva-based cortisol levels from speech (Spearman ρ 0.770). Speech-based deep learning models successfully predicted physiological markers of stress, yielding a Spearman correlation coefficient of 0.770 for cortisol measurements taken 10 minutes after a stress period.