A multimodal machine learning architecture combining physiological and brain activity reliably distinguished two different levels of mental effort in a cross-subject classification.
Observational (n=18)
A multimodal machine learning architecture combining physiological and brain activity data can reliably decode different levels of mental effort across individuals.
Humans' performance varies due to the mental resources that are available to successfully pursue a task. To monitor users' current cognitive resources in naturalistic scenarios, it is essential to not only measure demands induced by the task itself but also consider situational and environmental influences. We conducted a multimodal study with 18 participants (nine female, M = 25.9 with SD = 3.8 years). In this study, we recorded respiratory, ocular, cardiac, and brain activity using functional near-infrared spectroscopy (fNIRS) while participants performed an adapted version of the warship commander task with concurrent emotional speech distraction. We tested the feasibility of decoding the experienced mental effort with a multimodal machine learning architecture. The architecture comprised feature engineering, model optimisation, and model selection to combine multimodal measurements in a cross-subject classification. Our approach reduces possible overfitting and reliably distinguishes two different levels of mental effort. These findings contribute to the prediction of different states of mental effort and pave the way toward generalised state monitoring across individuals in realistic applications.
Gado et al. (Thu,) conducted a observational in Mental effort (n=18). Multimodal machine learning architecture was evaluated on Decoding experienced mental effort. A multimodal machine learning architecture combining physiological and brain activity reliably distinguished two different levels of mental effort in a cross-subject classification.