Stress classification with in-ear heartbeat sounds

Although stress plays a key role in tinnitus and decreased sound tolerance, conventional hearing devices used to manage these conditions are not currently capable of monitoring the wearer’s stress level. The aim of this study was to assess the feasibility of stress monitoring with an in-ear device. In-ear heartbeat sounds and clinical-grade electrocardiography (ECG) signals were simultaneously recorded while 30 healthy young adults underwent a stress protocol. Heart rate variability features were extracted from both signals to train classification algorithms to predict stress vs. rest. Models trained and tested using in-ear heartbeat sounds appeared to perform better than the models trained and tested using the ECG signals. However, further analyses comparing heart rate variability features extracted from ECG and the in-ear heartbeat sounds suggest that the improvement in stress prediction performance was driven by the increased presence of artifacts (e.g. movement or speech) during the stress tasks, rather than physiologically meaningful changes in the heartbeat signals that would be indicative of stress in real-world settings. To address this difference in error between rest and stress conditions, a data augmentation method was proposed to balance the error. The final system demonstrates the viability of robust stress recognition with only in-ear heartbeat sounds, which could expand the capabilities of hearing devices used to address conditions related to stress and noise. The proposed data augmentation method effectively identified and addressed artifact-related biases, which could broadly be applied to improve robustness of biosignal monitoring with machine learning. • Robust stress classification achieved using only in-ear heartbeat sounds. • Findings suggest the stress model exploited the error in in-ear audio features. • Increased feature error in stress conditions can lead to misleading accuracy scores. • Data augmentation method proposed to balance errors in stress and rest conditions. • Results could be applied to develop hearables that reduce stress from ambient noise.