What question did this study set out to answer?

The research aims to develop a framework using VR and EDA sensing to detect mild cognitive impairment non-invasively and effectively.

March 23, 2026Open Access

Integrating Virtual Reality, Electrodermal Activity and Machine Learning for the Early Detection of Mild Cognitive Impairment

Key Points

The research aims to develop a framework using VR and EDA sensing to detect mild cognitive impairment non-invasively and effectively.
Developed an MCIVR application using virtual environments for cognitive tasks and physiological data collection.
Used Oculus Quest 2 and Fitbit Sense 2 for capturing multimodal physiological signals.
Conducted a usability study with 50 participants to evaluate system performance and user engagement.
Employed Explainable AI techniques, like SHAP, for data interpretability and risk insights.
Found that the combination of VR and EDA features significantly improved detection of mild cognitive impairment.
Participants rated the MCIVR system above industry usability standards, indicating its effectiveness and engagement.
Physiological responses varied with different virtual environments, revealing important markers related to stress and memory.
Identified key design factors such as posture, environment, and calibration for successful cognitive assessments.

Abstract

Mild Cognitive Impairment (MCI) is an early stage between normal ageing and dementia, where recognising subtle changes in memory and thinking can enable timely treatment and lifestyle interventions. This thesis develops and evaluates an integrated framework that combines Virtual Reality (VR), Electrodermal Activity (EDA) sensing, and Machine Learning (ML) to support early, non-invasive, and user friendly detection of cognitive decline (CD). The system comprises two components: the MCIVR application, which uses vir tual environments to engage participants in cognitive tasks while measuring physiological signals, and the MCIFR framework, which applies Explainable Artificial Intelligence (XAI) to interpret these signals and generate personalised risk insights. The MCIVR system was iteratively refined through calibration sessions and Unity-based testing to ensure accurate physiological capture and reliable task interaction. A usability study with 50 ethically approved participants employed Oculus Quest 2 and Fitbit Sense 2 sensors to record multimodal data spanning physiological arousal, interaction performance, and self-reported workload. These datasets were benchmarked against Alzheimer’s Disease Neuroimaging Initiative (ADNI) data to validate predictive performance. Data processing included preprocessing, feature engineering, and classification within an XAI pipeline, using SHapley Additive ex Planations (SHAP) for interpretability. Findings show that combining VR and EDA features improves MCI detection compared with single-modality models. Participants rated the MCIVR system above industry usability standards, confirming feasibility and engagement. Physiological responses varied with the virtual environment, and explainability analyses high lighted clinically plausible markers, such as stress- and memory-related features. The study identifies posture, environment, and calibration precision as important design factors for effective VR-based Cognitive Assessment (CA). It demonstrates the potential of affordable consumer devices to support multimodal screening for cognitive decline. Overall, this research advances the integration of VR, EDA, and ML technologies and offers a reproducible, interpretable, and practical pathway towards earlier and more accessible cognitive health assessments.

Mark Helpful

Bookmark

Relay

View Full Paper