Robust multimodal mental workload classification: A cross-physiological condition machine learning approach

• Sleep restriction decreases performance of mental workload levels classification • Hypoxia decreases performance of mental workload levels classification • Evaluation of cross physiological conditions is important • Training models under different physiological conditons improves their robustness • Multimodal datasets improve models performances Aircraft pilots can be faced with a high mental workload (MW) combined with moderate hypoxia and sleep restriction. We aimed to assess the cross-validation of a machine learning-based MW predictive model under hypoxia and/or sleep restriction. Secondly, we developed a robust predictive model using multimodal physiological parameters to improve the validity across different physiological conditions. Seventeen healthy participants were randomly exposed to three 12-minute periods of increased MW (low, medium, and high) in a 4-condition crossover design: sleep restriction (SR, <3h Total Sleep Time, TST) vs. habitual sleep (HS, >6h TST), hypoxia (HY, 2h, F I O 2 =13.6%, ∼3,500 m) vs. normoxia (NO, F I O 2 =21%). MW levels were designed using the Multi-Attribute Test Battery (MATB)-II with an additional auditory Oddball-like task. Six machine learning classifiers were compared. Features selection (from EEG, ECG, respiratory and eye tracking sensors) was performed using backward Recursive Feature Elimination (RFE). The best models for 1-minute MW levels classification on HSNO were K-Nearest Neighbors (KNN, F1 score = 80.3±8.9%), Support Vector Machine (SVM, 77.8±10.3%) and Random Forest (RF, 75.7±9.1%). Exposure to sleep restriction and/or hypoxia decreased models’ performance (F1 <35%). KNN and RF models, in particular those including EEG and eye tracking, trained on All-Conditions performed well across conditions (F1 scores = 77.4±7.8% and 70.7±10.2%). Our results highlight the need for training MW models under different physiological constraints and using multimodal datasets to improve robustness. (NCT05563688)