Causal AI: Evaluating AI Workflows on HPC Environments Using Causal Testing

As AI and machine learning become central to scientific research, the ability to rigorously verify their behaviour has struggled to keep up. Traditional software testing breaks down in this context: scientific pipelines rarely have a single correct output to test against, their configuration spaces are vast, and their stochastic nature makes it difficult to distinguish genuine errors from expected variation. The consequence is that many scientific AI systems are deployed with limited understanding of what actually drives their outputs. Causal testing offers a principled alternative. By encoding domain knowledge as a Directed Acyclic Graph (DAG) - a structured representation of the cause-and-effect relationships between variables, such as batch size or floating point precision - researchers can formally test whether their pipeline behaves as expected, using data already collected from prior runs. This removes the need for expensive re-experimentation and makes verification tractable even at HPC scale. This poster presents causal-ai, an open-source tool developed as part of the N8 CIR AI4Science project to improve the ability of evaluating complex AI/ML workflows on HPC systems. The tool provides direct HPC integration, a command-line interface, and worked case studies across image classification and video domain adaptation pipelines. Cross-cluster comparisons further reveal platform-dependent effects that conventional evaluation metrics do not capture.