Immersive virtual simulation use as an experimental tool for recording speech production for machine learning

Speech-based stress detection is an emerging area in computational science and machine learning (ML), yet progress is constrained by the lack of large-scale, ecologically valid, and multimodally annotated speech-under-stress datasets. In this perspective article, we argue that immersive virtual reality (iVR) offers a promising methodological framework for addressing this gap. Building on evidence that virtual environments can elicit responses comparable to those in real-world settings, we outline five design principles for iVR-based speech corpora. First, iVR can help reconcile experimental control with ecological validity by enabling realistic, repeatable stressors and context-rich scenarios that elicit stress-modulated speech. Second, iVR paradigms can be designed to capture time-locked changes in speech and physiology around scripted events, informed by predictive processing accounts of cognition. Third, iVR platforms allow temporally aligned multimodal measurement, providing the data needed to test whether fusing speech with physiological and behavioral signals makes stress detection more robust and responsive. Fourth, the digital, shareable nature of iVR protocols can support scalable, cross-site, and cross-cultural data collection. Fifth, ethical safeguards concerning participant wellbeing, privacy, and potential misuse must be treated as core design constraints. Throughout, we distinguish findings supported by previous studies from methodological advantages that iVR offers in principle and from hypotheses about ML performance that still require empirical validation. We therefore outline requirements for ML-ready datasets and benchmarks, including validated multi-indicator stress labels, measured end-to-end latency, subject-independent data splitting, and cross-device, cross-scenario, and iVR-to-real-world evaluation.