Robot-assisted evaluation of vocal emotion recognition

Vocal emotion recognition can be a challenge for young children and in hearing impairment, and maintaining participant engagement is crucial to ensure reliable results. Human–robot interaction has been proposed in literature to help provide an engaging experience during repetitive tasks. We propose the use of a humanoid NAO robot as an interactive interface for a vocal emotion recognition (EmoHI) test, originally designed as a serious game. The test has 36 pseudospeech trials from four speakers expressing three emotions (happy, angry and sad). The test was conducted with English speaking normal-hearing adults (Experiment 1; n = 28, 19–36 years), and Spanish speaking cochlear-implanted adolescents (Experiment 2; n = 40; 10–17 years) twice, once via the robot and once via a computer. Test results (sensitivity index, d’) and durations were compared between interfaces. Engagement was assessed via questionnaires filled in after each test round. d’ results were similar between the computer and the robot (Experiment 1: 1.97 ± 0.39 vs. 2.02 ± 0.26; Experiment 2: 0.36 ± 0.36 vs. 0.37 ± 0.43). Test durations were similar in Experiment 1, but significantly longer on the robot in Experiment 2. Participants overall preferred the robot over the computer.