What question did this study set out to answer?

The research investigates how different tool serialization architectures affect cache economics in multi-tenant LLM systems.

May 20, 2026Open Access

Do LLM Agents Need Tool Schemas in Every Session? Cache-Aware Tool Use in Multi-Tenant LLM Systems

Key Points

The research investigates how different tool serialization architectures affect cache economics in multi-tenant LLM systems.
Defined six tool-serialization architectures and provided measurements for three (A, B, D) across multiple tenants.
Evaluated cache hit rates and cost implications using a closed-form cost model.
Developed a framework for comparing cache behavior across major LLM providers.
Delegation architecture achieves a prompt-cache hit rate of 98.4%; naive direct injection reaches only 74% with higher costs.
Demonstrated that cache-aware direct injection fails to maintain optimized cache geometry with increased tenant counts.
Introduced a decision framework matching deployment characteristics to architecture recommendations.

Abstract

Multi-tenant LLM applications often prepend user- or tenant-specific context before large tool schemas. Under longest-prefix prompt caching — the dominant cache discipline in modern LLM serving — this ordering fragments cache reuse: the schema block may be byte-identical across tenants but still fails to share cache entries because divergence occurs before the schema begins. This report studies tool serialization as a cache-economics problem: how prompt-construction choices determine whether large schema blocks amortize across tenants or are repeatedly billed. We define six tool-serialization architectures — naive direct injection (A), cache-aware direct injection (A′), provider-native tool APIs (Aₙative), top-m schema retrieval (B), goal-delegation broker (D), and broker with internal retrieval (Dᵣag) — and provide preliminary measurements for three of them (A, B, D) on DeepSeek across K ∈ 5, 10, 50, 200 personalized tenants over 200 tools and 3 sequential calls per tenant. Delegation maintains a 98. 4% prompt-cache hit rate independent of K; naive direct injection plateaus at 74% and incurs a 2. 10× higher total cost at K=200. We further derive a closed-form cost model parameterized by tenant count K, schema size S, personalization size P, cached-token discount α, tool-use rate q, and per-tenant catalog overlap μ, and prove a cache-geometry theorem characterizing when schema-region miss cost is Θ (S) versus Θ (K·S). Under per-tenant tool subsets — the prevailing real-world case in multi-tenant SaaS — cache-aware direct injection cannot preserve Θ (S) cache geometry in tenant count, while goal-delegation with a unified broker catalog plus runtime ACL can. The paper also contributes a cross-provider cache-behavior evaluation framework comparing Anthropic, OpenAI, DeepSeek, and self-hosted vLLM, and a decision framework mapping deployment characteristics to recommended architecture. Seven falsifiability conditions are pre-registered. This work extends the substrate-thesis introduced in Skill as Memory, Not Document (Sarkar 2026, doi: 10. 5281/zenodo. 20128887). The present report is preliminary; cross-provider replication, BFCL accuracy evaluation, and per-tenant subset sweeps are in progress.

Read Full Paperexternally

اسأل الذكاء الاصطناعي

Bookmark

View Full Paper