Token Velocity, introduced by the TokenScale system for autoscaling disaggregated large language model serving, is defined as the maximum number of tokens an instance can release per second under its current resources, measured separately for the prefill and decode stages and, for decode, separately per request shape. This paper applies that metric and its offline estimation procedure to hardware where it had not been measured: a single Google TPU v5e chip serving the Gemma 2B model through vLLM on the JAX based TPU software stack. The serving configuration is the optimized one established in the first two parts of this series, which raised throughput on this chip by a factor of 12.7 through a token selection correction, batch scaling, and a retuning of the attention kernel block size. Three results are reported. First, a nine bucket velocity table for the chip: prefill velocity is approximately 38 thousand input tokens per second and nearly constant across input lengths, while decode velocity varies from roughly 8.3 thousand output tokens per second for short inputs to 2.9 thousand for inputs near the context limit, governed almost entirely by input length and almost not at all by output length. Second, a controlled repetition of the entire table under the retuned attention kernel: the same change that gains up to 84 percent of decode velocity on short input buckets loses 24 percent on long input buckets, locating the regime boundary of the kernel tuning between 1,024 and 7,168 input tokens and confirming, with a measured sign change, the regime dependence argued qualitatively in the earlier parts. Third, a consequence for the metric itself: decode velocity is not a constant of a hardware and model pair, as autoscaling systems assume when they profile it once, but a property of the hardware, model, and kernel configuration triple; a kernel change deployed after profiling shifts the table in both directions at once. A worked fleet sizing example shows the kernel choice changing the required chip count for a fixed traffic mix. Part 3 of a measurement series; Parts 1 and 2 are available at DOI 10.5281/zenodo.21212010.
Rubens de Almeida Zimbres (Mon,) studied this question.