What is the clinical evidence from this study?

Study design: Other. Population: Healthy participants (emotion prediction) (n=65). Intervention: Bayesian deep learning framework (LSTM+CNN) vs. Non-Bayesian models and previous benchmarks. Primary outcome: Classification accuracy of emotional valence (high/low).

March 20, 2020Open Access

A Bayesian Deep Learning Framework for End-To-End Prediction of Emotion From Heartbeat

Key Result

An end-to-end Bayesian deep learning framework achieved a peak classification accuracy of 90% for predicting emotional valence from unimodal heartbeat time series.

Structured PICO

Does a Bayesian deep learning framework using unimodal heartbeat time series accurately predict emotional valence?

Population

65 healthy participants from two established datasets (AMIGOS: n=40, mean age 28.3, 13 female, 27 male; DREAMER: n=25, mean age 26.6, 11 female, 14 male).

Intervention

End-to-end Bayesian deep learning model (CNN + bi-directional LSTM with Monte Carlo dropout) using unimodal heartbeat time series (inter-beat-intervals).

Comparator

Non-Bayesian models, separate CNN/LSTM streams, and previous benchmarks (e.g., SVM, Naive Bayes).

Outcome

Classification accuracy of emotional valence (high vs. low).

A Bayesian deep learning framework using unimodal heartbeat time series can accurately predict emotional valence with quantifiable uncertainty, enabling potential applications in continuous non-invasive emotion monitoring.

Limitations

Extracting inter-beat-intervals from ECG markedly reduces the information content of the input signal compared to multimodal approaches.
As the certainty threshold increases, model coverage decreases because fewer predictions meet the threshold.
Reliance on subjective self-reporting of valence by participants.

Abstract

Automatic prediction of emotion promises to revolutionise human-computer interaction. Recent trends involve fusion of multiple data modalities -- audio, visual, and physiological -- to classify emotional state. However, in practice, collection of physiological data ‘in the wild’ is currently limited to heartbeat time series of the kind generated by affordable wearable heart monitors. Furthermore, real-world applications of emotion prediction often require some measure of uncertainty over model output, in order to inform downstream decision-making. We present here an end-to-end deep learning model for classifying emotional valence from unimodal heartbeat time series. We further propose a Bayesian framework for modelling uncertainty over these valence predictions, and describe a probabilistic procedure for choosing to accept or reject model output according to the intended application. We benchmarked our framework against two established datasets and achieved peak classification accuracy of 90 percent. These results lay the foundation for applications of affective computing in real-world domains such as healthcare, where a high premium is placed on non-invasive collection of data, and predictive certainty.

Bookmark

View Full Paper