Causality-Driven Test Case Minimisation for Cyber-Physical Systems

Cyber-physical systems allow digital control systems to interact with the physical world using sensors and actuators. They are increasingly being used to automate critical infrastructure, where software faults can have dire consequences. Due to the complex nature and unpredictability of these systems, their resilience is often tested using a technique called fuzzing, which generates quasi-random sequences of sensor and actuator manipulations with the goal of forcing a system into unsafe states. However, there is currently no way of determining which manipulations of a test case cause a failure without systematically removing each one and re-running the test, which can be extremely time-consuming and expensive. In this work, we present CausalCut, a technique that uses causal inference to estimate the causal contribution of each intervention from pre-existing runtime data, thereby reducing the number of times tests must be re-run. We evaluated CausalCut by applying it to two very different systems: an artificial pancreas and a water treatment plant. CausalCut typically managed to remove more than half of the spurious manipulations using fewer executions than the current state of the art, which represents a saving of up to 18 hours and 6300 per test case.