Machine Learning Cyberattack and Defense Strategies

Cybersecurity is an increasingly important challenge for computer systems. In this work, cyberattacks were modeled using an extension of the well-known Petri net formalism. That formalism, designated Petri nets with players, strategies, and costs, models the states of the cyberattack and events during the attack as markings and transition firings in the net respectively. The formalism models the attacker and defender as competing players who may observe the marking of a subset of the net and based on the observed marking act by changing the stochastic firing rates of a subset of the transitions in order to achieve their competing goals. Rate changes by the players incur a cost. Using the formalism, nets were constructed to model specific cyberattack patterns (cross-site scripting and spear phishing) documented in the Common Attack Pattern Enumeration and Classification database. The models were validated by a panel of cybersecurity experts in a structured face validation process. Given those validated nets, a reinforcement learning algorithm using an -Greedy policy was implemented and set to the task of learning which actions to take, i.e., which transition rates to change for the different observable markings, so as to accomplish the goals of the attacker or defender. Experiments were conducted with a dynamic (learning) attacker against a static (fixed) defender, a static attacker against a dynamic defender, and a dynamic attacker against a dynamic defender. In all cases, the reinforcement learning algorithm was able to improve its performance, in terms of achieving the player’s objective and reducing the cost of doing so, over time. These results demonstrate the potential of formally modeling cyberattacks and of applying reinforcement learning to improving cybersecurity.