A Novel Overlapping Coefficient-Based Framework for Integrating Multimodal Physiological Signals to Infer Cognitive Strategies and Operator Performance in Human–System Interfaces

This is the authors' abstract. We don't add key points for this paper.

In digitalized plants, control room operators experience cognitive overload, and literature emphasizes that multimodal physiological integration can better capture operators’ cognitive states. In chemical process operations, current methods often overlook cross-modal interactions. This study used a formaldehyde production simulation with 42 participants exposed to failure scenarios, assessing performance by recovery time and plant status. A novel framework for multimodal physiological integration is proposed, modeling high/low levels of eye-based, skin-related, and cardiovascular metrics using Gaussian distributions. Unique combinations of these metrics are formed, and the overlapping coefficient (OVL) is computed to identify consistent physiological combinations across participants. High-OVL combinations appeared in all optimal, 79% of good, and were negligible in the poor class. Successful participants exhibited distinct cognitive strategies, from low-arousal focus to high-arousal compensation. The Bayesian network estimated participants’ performance-level probabilities, achieving 91% accuracy and robustness to missing data. The framework supports reflective learning, supervisory support, and adaptive training systems.