Human-in-the-Loop Safety for MCP Database Servers

Large Language Models (LLMs) are increasingly granted direct access to enterprise data systems through the Model Context Protocol (MCP). This makes it possible for end-users to query operational databases in plain language and receive immediate, contextual answers. Such capabilities promise to democratize analytics and accelerate decision-making, but they also introduce a new class of high-consequence risks. A model can misinterpret ambiguous intent, overreach its role by touching unauthorized tables, or be coerced by carefully crafted prompts into revealing schema information or issuing destructive commands. In high-stakes domains such as finance, healthcare, or critical infrastructure, these risks are unacceptable without additional safeguards. In this work, we present a practical human-in-the-loop (HITL) safety layer designed specifically for MCP database servers. Our interceptor examines every SQL query generated by an LLM and assigns a risk score using a combination of lightweight heuristics (e.g., keyword patterns, schema sensitivity) and a calibrated machine-learning classifier. Queries deemed low risk are executed immediately to preserve responsiveness. Those in an intermediate “gray zone” are paused and surfaced to a reviewer through a lightweight web interface, while queries that match hard-deny patterns (e.g., destructive DROP or mass DELETE without predicates) are blocked outright. Reviewer outcomes—approve, modify, or reject—are continuously logged and fed back into the model, allowing the system to adapt and reduce unnecessary escalations over time. We evaluate the approach on a dataset of approximately 25,000 SQL-like queries (sql_- risk.csv), encompassing benign analytics, inadvertent model errors, and adversarial promptinjection attempts. Across thresholds, the hybrid system achieves 89–95% accuracy and maintains near-perfect recall for destructive queries, while reducing human intervention rates from roughly 20% at cold start to 7–10% with adaptation. Beyond quantitative results, we document case studies illustrating how the system intercepted schema exfiltration attempts, role overreach, and silent mass-update errors. We also provide an operational playbook, discuss ethical and privacy implications, and release reproducibility artifacts for community use. Our findings suggest that HITL checkpoints, when carefully engineered, can serve as a practical guardrail for deploying LLM agents in production data environments.