Answers are grounded in this paper and cite their sources.
Large Language Models (LLMs) provide strong capabilities across reasoning, coding, knowledge retrieval, and generation, but API usage becomes expensive when every request is routed to a high-cost model. The central problem is choosing an adequate model and inference strategy for each request while balancing quality, cost, latency, and operational constraints. This paper surveys twelve studies spanning unified routing infrastructure, benchmark design, cost-aware selection, preference-based routing, resource-aware serving, output-budget control, batching, cascading, quality-aware routing, adaptive test-time compute, and online routing with user feedback. The literature indicates a shift from static model choice toward multi-objective and adaptive routing. A practical gap remains for a lightweight, transparent, configurable application-level cost controller for small multi-provider applications. To address this, the paper proposes LLM Cost Autopilot, combining query analysis, explicit cost/capability rules, response caching, multi-provider model invocation, and cost/performance observability with an evaluation framework. No experimental results for the proposed system are claimed; it is presented as an implementation-oriented baseline for future extensions. Index Terms—LLM routing, cost optimization, cost-aware routing, model selection, response caching, token usage, latency, multi-model serving, adaptive compute, user satisfaction, AI cost monitoring.
No takes yet. Share an insight, caveat, or question.
Saxena et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: