Empathic communication is essential in clinical care but often burdens physicians, leading to empathy fatigue. To address this, systems that support emotional understanding and reduce cognitive load are needed. The aim of this study is to implement the examination room system pSCOUTER that we have developed and clarify its value from the perspective of physicians. pSCOUTER (pSychologic-Conditional Observing Unobtrusive Tracer Embedded Room) is a non-invasive system designed to observe patients’ emotional and psychological states without wearable devices, particularly in high-stress situations like cancer diagnosis. The system extracts physiological signals, contextual cues, and acoustic features such as tone and prosody, and estimates emotions using a cross-modal Transformer. Real-time feedback on the estimated emotional states helps physicians understand patients’ feelings, reduce empathy fatigue, and sustain empathic responses. In validating the effectiveness of the emotion estimation framework implemented in this system, we achieved an accuracy of 0.6548 and an F1 score of 0.6088 in an eight-class emotion classification task, exceeding the majority class baseline (Neutral: 0.3588) and clinicians’ estimates (accuracy = 0.3826, F1 = 0.2864). We implemented and evaluated this non-invasive system in real time. Furthermore, through multifaceted feedback from practicing physicians, we demonstrated the system’s value, clinical practicality, and acceptability.
Harima et al. (Wed,) studied this question.