Evaluation of an algorithm for optical pulse detection in children for application to the Pepper robot

To engage in socio-emotional interactions, children with autism spectrum conditions (ASC) need support to understand and convey emotions. In our approach, a humanoid robot (Pepper, Softbanks Robotics) acts as a tutor for the child within autism care. The robot, equipped with multimodal sensor technology to acquire the emotional feedback of the child, stimulates the child to perform tasks, adapted to its current arousal state. By in-, or decreasing the difficulties of implemented training modules, the child can be given the appropriate task according to its emotional state. The child’s arousal is measured with different techniques implemented in and on the robot: emotion detection based on audio recordings of the speech signal and camera detected facial expressions, or heart rate. To this end, the remote Photoplethysmography (rPPG) signal from camera recordings of the subjects’ face is acquired. While its unintrusive measurement is an advantage, a major drawback for rPPG is its proneness to motion and light artefacts requiring de-noising steps. A wavelet transform based on log-Gabor wavelets and a filter bank with 32 filters was implemented. The signal was filtered with a prior filter and afterwards with a Markov chain in order to extract the underlying pulse rate. Within an initial study, five children were observed watching videos with different co-notated emotions. As reference for the heart rate (HR), a wristband (empatica E4) was used. The captured emotions of all subjects were annotated to identify low and high arousal parts and positive and negative emotions. Extracted HR from rPPG-data indicated a correlation with the annotated emotions.