Preliminary report explores tool serialization's impact on cache efficiency in multi-tenant LLM applications, suggesting architectural frameworks.
Multi-tenant LLM applications often prepend user- or tenant-specific context before large tool schemas. Under longest-prefix prompt caching — the dominant cache discipline in modern LLM serving — this ordering fragments cache reuse: the schema block may be byte-identical across tenants but still fails to share cache entries because divergence occurs before the schema begins. This report studies tool serialization as a cache-economics problem: how prompt-construction choices determine whether large schema blocks amortize across tenants or are repeatedly billed. We define six tool-serialization architectures — naive direct injection (A), cache-aware direct injection (A′), provider-native tool APIs (A_native), top-m schema retrieval (B), goal-delegation broker (D), and broker with internal retrieval (D_rag) — and provide preliminary measurements for three of them (A, B, D) on DeepSeek across K ∈ {5, 10, 50, 200} personalized tenants over 200 tools and 3 sequential calls per tenant. Delegation maintains a 98.4% prompt-cache hit rate independent of K; naive direct injection plateaus at 74% and incurs a 2.10× higher total cost at K=200. We further derive a closed-form cost model parameterized by tenant count K, schema size S, personalization size P, cached-token discount α, tool-use rate q, and per-tenant catalog overlap μ, and prove a cache-geometry theorem characterizing when schema-region miss cost is Θ(S) versus Θ(K·S). Under per-tenant tool subsets — the prevailing real-world case in multi-tenant SaaS — cache-aware direct injection cannot preserve Θ(S) cache geometry in tenant count, while goal-delegation with a unified broker catalog plus runtime ACL can. The paper also contributes a cross-provider cache-behavior evaluation framework comparing Anthropic, OpenAI, DeepSeek, and self-hosted vLLM, and a decision framework mapping deployment characteristics to recommended architecture. Seven falsifiability conditions are pre-registered. This work extends the substrate-thesis introduced in Skill as Memory, Not Document (Sarkar 2026, doi:10.5281/zenodo.20128887). The present report is preliminary; cross-provider replication, BFCL accuracy evaluation, and per-tenant subset sweeps are in progress.
No takes yet. Share an insight, caveat, or question.
Pranab Sarkar (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: