Experimental evaluation demonstrates a 1.42× throughput gain and secure tenant isolation in an oversubscribed confidential GPU environment, indicating limits set by host RAM.
One confidential GPU serves a limited number of concurrent tenants. This report measures what happens when registration exceeds that number: idle tenants' key-separated KV state is parked to disk and revived on activation, on an Azure H100 confidential instance with SEV-SNP active and confidential compute attested at boot. At an activation period of 400 seconds, 512 registered tenants sustain a 2.00 s p95 time-to-first-token on revived requests, bound by page-cache residency. The honest gain over a no-parking baseline on the same harness is 1.42×, and at busy periods parking loses capacity rather than gaining it. Four interventions attempted to raise the gain — five generic codecs, fp8 KV quantisation, a 4× longer activation period, and a 3.4× smaller GPU KV arena. None raised it; two lowered it. The gain is structurally bounded by the ratio of host RAM to GPU KV arena. What does not shrink under correction is the isolation result. A 64-block known-byte canary was byte-identical in every cell of every arm to 768 registered tenants, rotated and unrotated; per-tenant rotation costs 0.54 percent of revival time; and a companion arc shows the same isolation surviving the engine's native preemption, recompute and eviction on the installed extension over stock vLLM. At these workloads the product is not the tenant count, which the hardware largely gives away, but the demonstrated safety of selling that count to tenants who do not trust each other. Every measurement was preregistered (P151–P219). Five corrections that moved headline figures during the campaign are reported with their magnitudes, the largest a factor of six.
No takes yet. Share an insight, caveat, or question.
Luis Carranza (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: