Multimodal Physiological Models of Situation Awareness

Future human-autonomy teams will require AI systems capable of adapting to human operators. Of all human states to adapt to, situation awareness (SA) is of particular interest due to its implications on performance and criticality for safety. SA comprises three levels: the perception of elements in the environment, the comprehension of their meaning, and their projection into the future. While existing measures of SA can be difficult to capture in real-time, physiological models have shown promise for predicting SA and could be implemented in real-time in a range of operational settings like air-traffic control. This information regarding operator SA could be used to develop autonomous teammates that respond to critical shifts in SA before adverse outcomes develop. This work advances prior efforts, by incorporating a broad combination of sensors to capture physiological variance in SA, leveraging three linear multiple regression models to predict the SA of an operator along a continuum at each of the three levels, and assessing generalizability of these models to new operators and unseen scenarios.