Automatically Assessing Children’s Internalizing and Externalizing Behavior From Face and Head Dynamics During Parent-Child Interaction

Research on the automated assessment of mental disorders has primarily focused on adult participants and on behaviors on the individual level. We propose an approach to automatically assess the severity of children’s behavioral, emotional, and social problems from videos of face-to-face parent-child interaction. Children’s behavioral, emotional, and social problems were quantified using the Child Behavior Checklist (CBCL), focusing on the two broad categories “internalizing” and “externalizing” and the more specific categories “anxious”, “withdrawn”, and “aggressive”. Our experimental data comes from a cohort of 81 8- to 10-year-old children and their parents. We constructed features to represent the nonverbal face and head behaviors of the parents and children, combined them with the children’s symptom scores, and then fed these data to binary classifiers to make broad estimations of symptom severity. Prediction performance was good only for anxiety scores, although the prediction of withdrawal and internalizing scores did show some promise as well. We moreover identified the behaviors that were most informative in the context of predicting anxiety and withdrawal and investigated how they were influenced by symptom severity and topic of conversation. Our results exemplify how machine learning and computer vision can be used to gain further insights into child psychopathology.