Validating a "Real-Time Assessment of Multidimensional User State" (RASMUS) for Adaptive Human-Computer Interaction

RASMUS ("Real-Time Assessment of Multidimensional User State") is a generic diagnostic framework designed to provide real-time diagnostics of user performance and up to six cognitive states impacting user performance. RASMUS diagnostics can be used in adaptive system designs to infer when the user needs support and to dynamically select and apply adaptation strategies that are suitable to restore the user's effectiveness in a given situation. In a proof-of-concept implementation, the generic RASMUS concept has been operationalized for a simulated air surveillance task. The implementation addresses three potentially problematic user states that are considered particularly relevant for this task domain: high workload, passive task-related fatigue, and incorrect attentional focus. This paper describes an experimental study aimed at validating the diagnostic capabilities of this proof-of-concept implementation. N=12 participants performed air-surveillance tasks in a scenario-based simulation. Congruent validity was assessed by comparing RASMUS diagnostics with subjective ratings of the respective user states. Ratings were obtained each time RASMUS detected a performance decrement. Outcomes showed that user states diagnosed as 'critical' by RASMUS were mostly consistent with subjective ratings at the time of performance decrements. However, a measure of Level 1-Situation Awareness proved to be more reliably associated with outcomes of RASMUS' "incorrect attentional focus" than a subjective measure of attention. Overall, outcomes support the validity of RASMUS diagnostics for the given task domain and the selected user states. Potential for further improvement is outlined.