Learning silhouette features for control of human motion

We present a vision-based performance interface for controlling animated human characters. The system combines information about the user’s motion contained in silhouettes from several view-points with domain knowledge contained in a motion capture database to interactively produce a high quality animation. Such an interactive system will be useful for authoring, teleconferencing, or as a control interface for a character in a game. In our implementation, the user performs in front of three video cameras; the resulting silhouettes are used to estimate his orientation and body configuration based on a set of discriminative local features. Those features are selected by a machine learning algorithm during a preprocessing step. Sequences of motions that approximate the user’s actions are extracted from the motion database and scaled in time to match the speed of the user’s motion. We use swing dancing, an example of complex human motion, to demonstrate the effectiveness of our approach. We compare the results obtained with our approach to those obtained with a set of global features, Hu moments, and to ground truth measurements from a motion capture system. Categories and Subject Descriptors: I.3.7 [ Computer Graphics ]: Three Dimensional Graphics and Realism— animation ; H.5.2 [ Information Interfaces and Presentation ]: User Interfaces— Input devices and strategies, Interaction styles ; G.3 [ Artificial Intelligence ]: Learning; I.4.8 [ Image Processing and Computer Vision ]: Scene Analysis— Motion, Tracking Our several users in street clothes and the resulting of an couple. For evaluation, we capture video and motion capture data simultaneously and compare the motion computed by our system with the ground-truth motion capture data. We also compare the performance of our system with that of a used set of global features, moments to demonstrate the of our local for