Timing control of utterance and gesture in interaction between human and humanoid robot

Towards establishing future-oriented relations between humans and robots, not only verbal, but also nonverbal robot functions have been studied. However, it is not clear how these different functions should be combined and implemented in robots in order to achieve `natural' human-robot communication or interaction. In this paper, we focus on the timing for the coordination between utterance as a verbal function and body motion as a nonverbal function. To begin with, we investigated the temporal order and correlation between utterance and gesture in an interpersonal (human-human) interaction scenario as a preliminary experiment (Experiment 1). Then, we controlled the robot's response timing based on findings of the first experiment and evaluated the effectiveness of the robot's timing control in human-robot interaction (Experiment 2). It was shown that elderly participants significantly preferred the controlled robot's response. In addition, the timing of utterance and gestures affected the impression of the robot and interpretation of the verbal message. These results are important findings for designing socially interactive robots for the aging society.