Spatial Interpretation of Pointing Gestures

Pointing gestures are an essential component of human communication, particularly when referring to distant objects, people, or events. Depending on the spatial characteristics of the environment, such gestures may complement verbal utterances or even replace them, especially in the absence of shared language or when the referent lacks salience and is thus difficult to describe. Although often perceived as more ambiguous in the latter case, observers are able to derive an interpretation. This thesis addresses the question of how people infer the target location a pointing gesture indicates without available verbal descriptions. The three studies of the present dissertation demonstrate that interpreting pointing gestures is more complex than often assumed. The key findings across all experiments indicate that interpretation relies on two distinct cues – namely the arm direction cue and the fingertip position cue –, which rarely indicate the same location. The relative weight of these cues within the final interpretation is determined by the spatial configuration between pointer and observer (Study I). Beyond this perspective dependency, vertical and horizontal information conveyed by the gesture is not integrated additively. Instead, it interacts in a way that causes identical gestures in one dimension to be assessed differently depending on the information conveyed in the other dimension (Study II). This interaction pattern is remarkably consistent across observers. While the index finger provides the primary information, the orientation of the arm is contrasted with that of the finger and also significantly affects the interpretation (Study III). In general, straight gestures are perceived as more linear than bent ones, although this cannot be attributed to increased perceptual certainty due to a larger stimulus. Furthermore, the studies replicate the finding that vertical extrapolation over distance is non-linear and extend this observation to horizontal interpretations (Studies II & III). The findings are discussed with emphasis on the non-linearity of pointing interpretation and within the greater picture – including the spatial environment as well as the communicative context – in which such gestures naturally occur. This thesis offers a detailed multi-step model of the process from perceiving a pointing gesture to deriving the final interpretation. Future research could build on these insights by examining the role of self-relevance for the observer in spatial interpretation, as well as the dynamics and mutual adaptations of pointer and observer during a single pointing episode or across extended interactions.