CAP: Detecting Network Device Misconfigurations with Context-Aware Prompting of LLMs

Verifying network configurations is critical for ensuring operational stability, but traditional tools force a trade-off between the high manual effort of model checkers and the semantic shallowness of consistency checkers. More importantly, neither class of tools offers interpretable, human-readable explanations to support diagnosis. Large Language Models (LLMs) present a promising alternative by enabling both detection and explanation. However, naive approaches like full-file, partition-based, or chain-of-thought prompting fail to handle the large, interconnected nature of configuration files. We introduce the Context-Aware Prompting (CAP) framework, which enables an LLM to reason about network configurations like a human expert. CAP first identifies all potentially relevant neighboring, similar, and referenced configuration segments. It then engages the LLM in a structured dialogue, where the model requests the specific context it needs before performing a focused analysis. This on-demand approach provides the necessary context for deep reasoning while avoiding overload. Our evaluation on production network configurations demonstrates that CAP reliably detects a wide range of known and previously unknown configuration errors. CAP substantially outperforms existing LLM-based baselines, while achieving performance comparable to state-of-the-art non-LLM tools. CAP is most effective on misconfigurations that require reasoning over references and intricate dependencies.