Deterministic and Auditable Routing for Artificial Intelligence in Regulated Environments

# Deterministic and Auditable Routing for Artificial Intelligence in Regulated Environments **Felippe Barcelos** Independent Researcher felippe. barcelos10@gmail. com *Preprint — submitted to Zenodo. Version 1. 0. 0 — Protocol P55. 0 — 2026-06-03* --- ## Abstract The deployment of Large Language Models (LLMs) in regulated industries — banking, healthcare, and government — demands audit trails that satisfy strict reproducibility requirements under the EU AI Act (2024), the General Data Protection Regulation (GDPR), and sector-specific frameworks such as HIPAA and Basel III/SR 11-7. Existing ML-based routing systems, including RouteLLM and FrugalGPT, achieve significant cost reductions but produce non-reproducible routing decisions whose outputs change with model retraining, rendering them incompatible with formal compliance auditing. We present the **TEIA Cognitive Router**, a compliance-first LLM routing system based on a fixed six-axis semantic entropy formula that produces deterministic routing decisions without neural weights, training data, or external dependencies. The system guarantees the *Write==Read invariant*: identical input text always yields an identical routing decision, an identical canonical JSON representation, and an identical SHA-256 audit seal. Routing decisions are organized in an append-only Merkle time-anchor chain and can be notarized by any RFC 3161-compliant Trusted Timestamp Authority (TSA) for legally binding external proof of existence. Evaluation on a 100-prompt simulation aligned with MT-Bench benchmark categories demonstrates **99. 6% quality retention with 16. 3% cost reduction** in compliance-safe mode, and **73. 8% quality retention with 98. 4% cost reduction** in max-savings mode. We provide a formal compliance mapping to EU AI Act Articles 12, 13, and Annex IV; GDPR Article 22; SOC 2 CC7; and HIPAA §164. 312 (b). The system is released under Apache 2. 0 and is available on PyPI as `teia-cognitive-router`. **Keywords: ** LLM routing, compliance, determinism, audit trail, EU AI Act, GDPR, SHA-256, Merkle chain, RFC 3161, semantic entropy **arXiv classifications: ** cs. CR (Cryptography and Security) · cs. AI (Artificial Intelligence) · cs. LG (Machine Learning) --- ## 1. Introduction The deployment of Large Language Models (LLMs) across regulated industries has accelerated markedly since the public release of instruction-tuned models beginning in 2022. Financial institutions leverage LLMs for risk analysis, contract review, and fraud detection. Healthcare providers deploy them for clinical documentation, drug-interaction queries, and patient communication. Government agencies apply them to policy analysis, benefits adjudication, and citizen services. This acceleration has collided with a fundamental regulatory barrier: the *audit reproducibility problem*. Modern regulatory frameworks — the European Union AI Act (2024), GDPR (2018), HIPAA (1996), and the Federal Reserve's SR 11-7 model risk guidance — require that automated decision-making systems be *auditable*, *reproducible*, and *explainable*. An organization must be able to answer, for any past automated decision: "Why did this happen? Can you prove it happened exactly this way? Has the record been tampered with? " LLM deployments routinely employ multi-tier model architectures to optimize cost: a small, fast local model handles simple tasks; a large, expensive cloud model handles complex tasks. The routing decision — which tier receives each request — is itself an AI-driven automation. Yet this decision is typically made by an opaque ML classifier (RouteLLM 1), a learned cascade (FrugalGPT 2), or an informal heuristic with no formal audit trail. When a compliance officer asks for proof of how a specific routing decision was made six months ago, the organization cannot provide a mathematically verifiable answer. This paper makes the following contributions: 1. **TEIA Cognitive Router**: A fixed arithmetic routing formula that produces deterministic, verifiable routing decisions without neural weights, training data, or external calls, with a formal proof of the Write==Read invariant (Section 4). 2. **Three-Layer Cryptographic Audit Architecture**: SHA-256 body seal + append-only Merkle time-anchor chain + RFC 3161 trusted timestamp, with formal tamper-detection proofs (Section 5). 3. **Compliance-Safe Mode**: A conservative threshold (E (t) < 0. 20) that achieves 99. 6% quality retention with a risk-free 16. 3% cost reduction (Section 6). 4. **Policy Engine**: A JSON-specified rule layer that applies hard compliance overrides (HIPAA, GDPR data locality) on top of the entropy verdict, with its own deterministic policyₛeal (Section 4. 4). 5. **Regulatory Mapping**: A formal analysis of TEIA's compliance properties against EU AI Act, GDPR, SOC 2, and HIPAA (Section 7). The remainder of this paper is structured as follows. Section 2 reviews background and related work. Section 3 analyzes the compliance failures of ML-based routers. Section 4 presents the TEIA system design. Section 5 describes the cryptographic audit architecture. Section 6 reports empirical evaluation. Section 7 provides regulatory analysis. Section 8 concludes. --- ## 2. Background and Related Work ### 2. 1 LLM Cost Optimization via Routing The cost of frontier LLM inference is substantial and highly variable by task. GPT-4-class models charge approximately 0. 010–0. 030 per 1, 000 tokens, while locally hosted 7–13B parameter models incur only compute costs of approximately 0. 0004–0. 001 per 1, 000 tokens on commodity hardware. For organizations processing millions of requests daily, routing decisions have direct revenue implications. **RouteLLM** (Ong et al. , 2024) 1 frames routing as binary classification: given a prompt, should it route to a "strong" or "weak" model? They train classifiers — including a causal LLM preference model and a matrix factorization model — on preference data from human feedback, achieving 40–50% cost reduction relative to a strong-model baseline on MT-Bench while maintaining quality above the 95th percentile. **FrugalGPT** (Chen et al. , 2023) 2 employs a learned cascade: each model in a chain has a learned scoring function and prompts halt when the score exceeds a learned confidence threshold, achieving 98% quality at 40% of cloud cost on representative workloads. Both systems represent significant engineering achievements in LLM cost optimization. However, Section 3 argues that their reliance on learned parameters renders them incompatible with formal compliance auditing. ### 2. 2 Merkle Trees and Cryptographic Audit Logs Merkle (1988) 7 introduced hash trees as a structure for efficient and secure verification of content in large datasets. Certificate Transparency (Laurie et al. , 2013) demonstrated that append-only Merkle logs can serve as tamper-evident public audit logs for TLS certificates. The TEIA temporal chain applies this principle to LLM routing audit records. ### 2. 3 RFC 3161 Trusted Timestamping RFC 3161 (Adams et al. , 2001) 5 defines the Internet X. 509 Public Key Infrastructure Time Stamp Protocol. A Trusted Timestamp Authority (TSA) cryptographically signs a hash along with a trusted time, producing a TimeStampToken that constitutes legally binding proof of existence under eIDAS Article 41 (qualified timestamps) and analogous national legislation. ### 2. 4 Compliance Frameworks The **EU AI Act** (Regulation 2024/1689) establishes requirements for high-risk AI systems, including comprehensive logging (Art. 12), transparency (Art. 13), and technical documentation (Annex IV). **GDPR** Article 22 restricts automated decision-making with legal or similarly significant effects, requiring explainability and the right to human review. **HIPAA** §164. 312 (b) mandates audit controls for systems handling protected health information. **Basel III/SR 11-7** requires model risk management including validation, documentation, and drift monitoring. --- ## 3. The Compliance Failure of ML-Based Routing ### 3. 1 The Reproducibility Requirement Formal compliance auditing requires *reproducibility*: given a historical record of a decision, an independent auditor must verify that the decision was made as claimed. **Definition 1 (Reproducible Routing). ** A routing function f is *reproducible* if: - (i) f (P) is a pure function of prompt P alone, with no dependence on mutable external state;- (ii) there exists a deterministic verification function verify (P, R) that returns True if and only if R = f (P) ; and- (iii) a collision-resistant commitment C (P, R) binds the stored record to the computation. ### 3. 2 Why Learned Routers Fail This Requirement **Non-reproducibility via weight drift. ** RouteLLM trains a classifier with parameters θ such that the routing decision is d = argmaxc P_θ (c | P). The decision is a function of both P and θ. After routine retraining (required for performance maintenance and concept drift), the updated parameters θ' may produce d' (P) ≠ d (P) for the same input P. An auditor attempting to reproduce a past routing decision from the input alone cannot recover the historical weights, making reproduction impossible in practice. The organization faces a choice between storing immutable historical model weights forever (operationally impractical) or being unable to prove past routing behavior (compliance failure). **Hash non-determinism. ** Because the routing function f_θ (P) is parameterized by mutable weights, the SHA-256 hash H (canonical (P || d) ) cannot serve as a verifiable commitment to a reproducible computation. The same prompt P may produce different decisions at different times. A stored hash proves only that P produced d at *some* point, not that P *deterministically implies* d. **Training data provenance. ** ML-based routers learn from examples of model performance on historical prompts. Under GDPR Article 22 and Article 13, automated decisions must be explainable. A routing model trai