Answers are grounded in this paper and cite their sources.
Human communication is highly adaptive: speakers continuously adjust how they speak, move, and look in response to changing communicative conditions. These adaptations are central to face-to-face interaction and are increasingly relevant for speech technologies that aim to operate in natural, interactive, and multimodal settings. While adaptive speech has been extensively studied within the articulatory and acoustic domains, spoken communication is fundamentally multimodal, raising the question of how visual and bodily cues participate in adaptive behaviour. Lindblom’s classic Hyper- and Hypo-articulation (H&H) Theory provides a principled account of adaptive speech production by framing variability as a trade-off between effort and intelligibility. However, how this adaptive logic extends beyond speech to multimodal communication remains insufficiently integrated across disciplines. This review synthesises research on speech and a range of visual and bodily signals to examine how multimodal behaviours contribute to adaptive communication. We first review evidence on how environmental, listener-related, and speaker-related constraints shape adaptive responses in speech. We then extend this analysis to multimodal communication, highlighting how speakers and listeners flexibly redistribute communicative effort across modalities in response to environmental, listener-related, and speaker-internal constraints. We argue that multimodal cues participate in the same adaptive optimisation described by the H&H Theory, and that adaptation emerges from dynamic trade-off across modalities rather than from speech alone. This multimodal reinterpretation of H&H has implications for speech technologies, including audiovisual speech, synthesis, and interactive systems. • Revisit Lindblom’s Hyper–Hypo Theory from a multimodal perspective • A review of adaptive speech and multimodal communication. • Environmental, listener, and speaker constraints are jointly considered. • Adaptation emerges from trade-off across communicative modalities. • Implications are discussed for audiovisual and interactive speech systems.
No takes yet. Share an insight, caveat, or question.
Charuau et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: