What question did this study set out to answer?

This research aims to identify the challenges and risks associated with the explainability of AI systems in OT cybersecurity environments.

May 18, 2026Open Access

Explainability Risks in AI-Driven OT Security

Key Points

This research aims to identify the challenges and risks associated with the explainability of AI systems in OT cybersecurity environments.
Analysis of AI-driven cybersecurity systems in ICS and OT environments.
Evaluation of governance and operational risks related to opaque AI decision-making.
Proposing a governance-oriented framework for explainable AI in OT cybersecurity.
Identified that lack of transparency in AI decisions creates challenges for trust and accountability.
Highlighted risks of override behaviors and governance failures due to mistrust in AI outputs.
Proposed a framework emphasizing transparency, auditability, and human-in-the-loop validation for AI in critical infrastructure.

Abstract

Author: Muhammad Ali Khan, ICS/ OT Cybersecurity Specialist and Researcher AAISM | CISSP | CISA | CISM | CEH | ISO27001 LI | CHFI | CGEIT This research examines the explainability challenges of Artificial Intelligence (AI)-driven cybersecurity systems in Industrial Control Systems (ICS) and Operational Technology (OT) environments, focusing on the governance, operational, and compliance risks associated with opaque or “black-box” AI decision-making. The study investigates how AI-based OT security systems increasingly influence cybersecurity responses in critical infrastructure while often lacking transparent reasoning for their outputs, creating challenges for trust, accountability, auditability, and operational safety. The research explores why explainability is particularly critical in OT environments compared to traditional IT settings, where security decisions may directly affect public safety, industrial continuity, and critical services such as power, water, and manufacturing. It evaluates the tension between statistical AI-based threat detection and the deterministic, cause-and-effect mindset of OT operators and engineers, highlighting the operational risk of mistrust, override behaviors, and governance failures.Additionally, the study proposes a governance-oriented framework for explainable AI (XAI) in OT cybersecurity, emphasizing transparent decision criteria, confidence scoring, auditability, human-in-the-loop validation, procurement requirements, and AI governance integration. The research argues that explainability should be treated as a foundational cybersecurity and governance requirement rather than a technical enhancement, particularly in regulated critical infrastructure environments.Overall, the research contributes to emerging scholarship on AI governance in critical infrastructure cybersecurity by reframing explainability as a prerequisite for accountable, trustworthy, and operationally resilient AI deployment in ICS/OT environments

Explainability Risks in AI-Driven OT Security

Key Points

Abstract

Cite This Study