Governing AI Voice Agents in Production: Latency, Safety and Evaluation Gates for Conversational AI Delivery

AI voice agents are moving from pilots into production customer operations, where they hold real-time spoken conversations with customers. They fail in ways text-based LLM systems do not: recognition errors cascade through the dialogue, response latency is experienced as rudeness or incompetence, and a hallucinated sentence spoken aloud carries an authority that text does not. This perspective paper extends the author's previously published governance framework for production LLM systems - observability, guardrails and evaluation gates operated inside the delivery cadence - to the specific physics of voice: per-turn latency budgets, speech-recognition confidence as a governed signal, spoken-output guardrails, simulated-conversation evaluation gates, and human escalation designed as a first-class deliverable. The paper proposes concrete, testable acceptance criteria and gate thresholds for voice-agent programs. It is a framework contribution, not a results study: no voice-agent deployment metrics are claimed, and every numeric budget given is labelled a design target for programs to validate in their own context.