Multimodal approach to situation awareness classification using physiological sensors

Accurately predicting operators' situation awareness (SA) is important in many work contexts. However, current well-validated SA measurement methods require task interruption, motivating alternative measurement approaches. We developed a multimodal ensemble model, using six different non-invasive physiological sensors, to predict high/low SA based on Endsley's three levels. We used a dataset of 31 participants performing the MATB-II, with periodic freeze-probe assessments about the task state to objectively measure SA. We fit logistic regression models for each sensor, combined them using a weighted average, and evaluated them on unseen data. This approach significantly outperforms various baselines, including shuffled-labels, random guessing, and constant-class predictions for all but level 3 SA. Level 3 was more challenging to predict as models did not significantly outperform the constant-class baseline. Sensor importance analysis identified electroencephalogram as consistently most important, followed by eye-tracking. These findings demonstrate effective high/low SA predictions with linear classifiers using physiological data.