Towards the design of multimodal interfaces for handheld conversational characters

This paper presents a study of individuals having conversations with animated characters on PDAs, and characterizes their use of natural nonverbal behavior compared to behavior exhibited in similar conversations with another person. The study finds that most people use the same nonverbal behavior in conversation handheld characters that they use in conversations with people, although the frequency is somewhat lower in the handheld case. These results can inform the design of new PDA input modalities which leverage the natural nonverbal behavior observed.