Automatic Collection of Scoring Metrics in Competitive Cybersecurity Lab Environments

Cybersecurity game-based learning has shown great promise and is likely to be a powerful means of training security professionals in the future.In most game-based cyber security learning systems there are automated scoring mechanisms in place that score participant performance based on some observable scoring metric.The scoring metric is derived from a predefined learning objective.In the Collegiate Cyber Defense System (CCDC) the services score is calculated by a periodic poling of a service to check that it is still online.Some cybersecurity game-based learning systems use sophisticated agents that collect heuristics on the participants' actions and compare them with the actions of known professionals.However, it is rare that the scores calculated automatically by the games system is the only score that a participant receives, often an observer also submits a score for each participant or group of participants.This paper discussed a system, designed using a five-step process that bridged the gap between a simple service poling score and highly sophisticated heuristics.The five steps were, continuous collection of scoring data, transportation, storage, analysis, and comprehensive display.The scoring system was designed to be used in a Competitive Labs-as-a-Service (CLaaS) learning system.The scoring system provides light-weight, secure, and automated scoring of specific data points as deemed necessary by predefined learning objectives.