Situation awareness (SA) in cyber-security is difficult to measure, yet new tools from research and industry promise improved cyber SA. This paper describes a human-subjects experiment using a high task fidelity cyber-security simulator. Participants from two groups (novice and experienced) were recruited and assigned to one of two interfaces (text and visual analytic), in a 2x2 between-subjects experimental design. The underlying cyber-security data presented to participants was the same for each interface, and included intrusion detection, firewall and vulnerability scan reports spanning the same time period. The participant’s situation awareness was measured using different techniques including a freeze-probe, post-trial assessments of perceived workload and perceived SA as well as a measure of task effectiveness. Comparison of the results indicate that this multi-method approach of cognitive assessment may be useful in understanding a phenomenon like SA, especially where tasks are complex and virtual such as is common in cyber-security.
Nicklaus A. Giacobe (Sun,) studied this question.