What counts as a relevant gesture in the study of multimodal event expressions?

This article explores how methodological decisions about which gestures to include impact the analysis of multimodal event representation. Gestures commonly temporally align with semantically co-expressive speech. However, when we consider not single concepts but whole events the picture gets more complicated, since events are not mapped onto single lexical items but across larger stretches of spoken language. To research gesture-speech integration in multimodal event expressions we must decide which gestures are considered ‘relevant’. Using caused motion events as a case study, we apply different definitions of gesture inclusion relating to temporal and semantic alignment. We investigate the impact of definition choice on the results by exploring corpus data from Australian English and two endangered Oceanic languages, Saliba-Logea and Sudest. The study shows comparable trends across some conditions but also language- and definition-specific differences. This has a bearing on how confidently we can compare studies based on different methodologies and/or languages.