Semantic complexity estimation and price-based multi-agent negotiation reduce tail latency and energy in edge-cloud generative inference

Serving large language models across mobile devices and cloud servers is hard because the cost of a generative query cannot be read off its input: a short prompt may trigger thousands of decoding steps. Schedulers therefore commit resources before knowing what a query will cost, and in a shared network many congest one server at once. In MAGS, each device runs a small probe that forecasts a query’s token count and generation entropy before decoding begins, then bids for cloud capacity against a congestion price broadcast by the server, selecting its per-token privacy parameter in the same bid under a metric differential privacy mechanism with explicit clipping. Two design choices follow from measurement. Splicing an 8B model’s lower layers into a 70B model’s upper layers destroys accuracy even with a trained adapter (24.7% versus 82.4% Pass@1 on GSM8K), so activation exchange stays within one model and the larger model is reached by prompt-level offload; and because migrating a key-value cache costs between 2.0 and 16.0 MB per decoder layer, the partition is fixed at prefill. Randomising the packet header cuts an eavesdropper’s identification of the source corpus from 71.1 to 24.7%. Across ten seeds MAGS lowers 99th-percentile latency by 17.1% and energy per token by 13.6% against a reactive PPO baseline given identical observations, and raises throughput by 20.7%; hardware testbeds reproduce the latency gain to within 0.6 percentage points on a single device and cut the correlation between eight devices’ offload decisions from 0.41 to 0.08. Beyond roughly 500 devices per server performance converges to the local-execution bound; doubling server capacity moves that point by the same factor, placing the limit in the shared server rather than in the pricing rule or the learner.