Understanding a child's play for robot interaction by sequencing play primitives using Hidden Markov Models

In this paper, we discuss a methodology to build a system for a robot playmate that extracts and sequences low-level play primitives during a robot-child interaction scenario. The motivation is to provide a robot with basic knowledge of how to manipulate toys in an equivalent manner as a human does - as a first step in engaging children in cooperative play. Our approach involves the extraction of play primitives based on observation of motion gradient vectors computed from the image sequence. Hidden Markov Models (HMMs) are then used to recognize 14 different play primitives during play. Experimental results from a data set of 100 play scenarios including child subjects demonstrate 86.88% accuracy recognizing and sequencing the play primitives.