What question did this study set out to answer?

This study aims to assess the effects of dynamically reducing numerical precision on thermal management in Mixture-of-Experts inference under radiative constraints.

June 13, 2026Open Access

Thermal-Adaptive Expert Precision for Orbital Mixture-of-Experts Inference: Measured Headroom, Measured Cost, and the Radiator-Bound Tail

Key Points

This study aims to assess the effects of dynamically reducing numerical precision on thermal management in Mixture-of-Experts inference under radiative constraints.
Evaluated expert numerical precision under thermal budget restrictions with a focus on INT8 and INT4 quantization.
Measured energy and accuracy impacts through predefined thresholds before analysis.
Utilized Grace–Blackwell hardware for performance metrics.
INT8 quantization resulted in +0.01% perplexity and 1.58× decode-energy reduction, shifting 48.9% of extreme configurations to feasible.
INT4 quantization yielded only 1.77× energy savings, below predictions, while increasing perplexity by 2.6–3.0%.
A persistent need for energy reduction in 9% of configurations, unsolvable by current software precision adjustments, was identified.

Abstract

In a companion study we showed that, for orbital Mixture-of-Experts (MoE) inference un- der absolute radiative thermal constraints, expert-placement optimization is nearly valueless: placement only redistributes a fixed heat total, and a thermally balanced placement is already near-optimal. Here we evaluate the lever that shrinks the total: dynamically reducing expert numerical precision in response to the thermal budget — a problem SpaceMoE 2 lists as open, and for which DynaExq 3 supplies the systems mechanism under memory (HBM) budgets. Our contribution is the thermal side: a pre-registered analytical headroom gate, measured accuracy, measured energy, and the resulting feasibility arithmetic for orbital serving. Three findings. (1) The INT8 lever is real and measured on both axes. Weight quantization to 8 bits costs +0.01% perplexity at full corpus on both Qwen3-30B-A3B and OLMoE-1B-7B — accuracy-free across a∼30×active-parameter range — and delivers a measured 1.58×(1.40×on the smaller model) decode-energy reduction on Grace–Blackwell hardware, clearing our pre-registered threshold; under the companion study’s feasibility model it flips 48.9% of thermally doomed (“physical-law”) configurations to feasible, at a median reduced-precision duty cycle of 0.547. (2) Modeled INT4 optimism does not survive measurement. A 4-bit deployment (Q4 K M) measures only 1.77×— far below the 2.95–5.45×our operation-count model predicted — because memory-bound decode pins savings near the bit-width floor while dequantization overhead consumes part of it; it adds little energy beyond INT8 while costing +2.6–3.0% perplexity. (3) A hard residual tail exists. The most thermally over-committed ∼9% of physical-law configurations require 3.25–4.60×energy reduction; no measured software precision lever reaches them, and they are radiator-sizing problems. The practical guidance is one sentence: quantize to INT8 on a thermal duty cycle — it is free; do not expect INT4 energy miracles in memory-bound serving; and size radiators for the tail. All thresholds were pre-registered before data, frozen in a separate commit, and never retuned.

Read Full Paperexternally

Perguntar à IA

Bookmark

View Full Paper