Auditing LLM Tool-Choice Signals: Geometry, Observability, and Conditional Validity

Objective: Determine whether emitted language-model tool-choice signals support claims about abrupt routing transitions, cost sensitivity, or error awareness. Methods: Audit route geometry, observability, and conditional validity using paired internal-only outcomes and outcome-blind baselines across multiplication, transfer tasks, and held-out HotpotQA/MBPP. Compare smooth and abrupt curve models and evaluate route and allocation signals at matched idealized tool-call budgets. Results: The archived multiplication 10%–90% routing region spans 1.1–1.6 digits, while smooth logistic curves predict held-out choices better than abrupt steps. Numeric-cost replications show no detectable crossing shift. In a held-out audit of 2,400 requests, the emitted route has AUROC 0.585 versus 0.770 for grouped complexity. Among 792 complete-case sources, complexity plus three-answer disagreement reaches AUROC 0.808 and 76.6% exact-tool counterfactual accuracy, versus 73.2% for the emitted route, but requires 1.38–1.42 times as many reported tokens. Limitations: Results are task-, model-, and prompt-conditioned; numeric cost markers are not realized prices or latency; some route and outcome cells are unobservable; and exact-tool evaluations are idealized counterfactual analyses. Conclusions: Tool-choice geometry, observability, and conditional validity must be established separately. S-curves and route labels alone do not establish a universal transition, latent utility, or metacognition.