System evaluation reveals systemic tool hallucination in frontier models during in-car voice tasks, highlighting the need for deterministic reliability kernels.
We present Autex, a reliability-first in-car voice assistant agent submitted to the CAR-bench Challenge at IJCAI-ECAI 2026. Autex combines a deterministic multi-layer reliability kernel with a configurable LLM verifier to address three core CAR-bench failure modes: hallucinating unavailable capabilities, acting on ambiguous user intent, and violating domain policies. Epistemic guards fire before any LLM call, reducing the consistency gap between Pass@3 (latent capability) and Pass∧3 (deployment-readiness). We validate on 369 official CAR-bench trials across four frontier models, identify hallucination missing tool response as the dominant systemic weakness (0–14.3% Pass∧3 across all models tested), and describe the design choices made to address it. All model and provider configuration is environment-variable-driven, satisfying the Track 2 auditability and reconfigurability requirements.
No takes yet. Share an insight, caveat, or question.
Eslam Abdelmogood Abdelkader (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: