Pointing to Facilitate Telepresence Robot-Mediated Participation in Hybrid Classroom Interaction

If a person in a video-mediated interaction (VMI) points to something, it can be very difficult for the other people to understand the gesture. This study investigates pointing gestures in a multi-party context in which a remote student participates in classroom interaction by using a telepresence robot, a remotely operated and movable VMI tool. We show how the pointing gestures by the “local” classroom participants involve (re)positioning their bodies to produce visual compositions designed to be understandable to the robot-mediated “remote” recipient and trace how the recipient orients to these gestures. By contrasting remote participants’ screen-recorded data with classroom video data, we also discuss how transmission delay (“lag”) may affect the temporal relationship between a gesture and other resources within an emerging multimodal action package and thus configure when exactly a gesture becomes visible to the recipient. Data are in Swedish and English.