Task‐level interaction with virtual environments and virtual actors

In many virtual environment (VE) applications, the VE system must be able to display accurate models of human figures that can perform routine behaviors and adapt to events in the virtual world. In order to achieve such adaptive, task‐level interaction with virtual actors, it is necessary to model elementary human motor skills. SkillBuilder is a software system for constructing a set of motor behaviors for a virtual actor by designing motor programs for arbitrarily complicated skills. Motor programs are modeled using finite state machines, and a set of state machine transition and ending conditions for modeling motor skills has been developed. Using inverse kinematics and automatic collision avoidance, SkillBuilder was used to construct a suite of behaviors for simulating visually guided reaching, grasping, and head‐eye tracking motions for a kinematically simulated actor consisting of articulated, rigid body parts. All of these actions have been successfully demonstrated in real time by permitting the user to interact with the virtual environment using a whole‐hand input device.