Heat-kernel context compression for transformer representations using graph diffusion

Introduction: Long contexts often contain geometrically redundant internal representations. Heat Kernel Context Compression (HKCC) is a training-free representation–compression method for transformer systems in which internal embeddings and attention logits can be modified. Materials and methods: Context representations are placed on a sparse similarity graph. The method uses the random-walk Laplacian throughout, while symmetric normalization appears only through an explicit similarity transform used for stable Chebyshev computation. Heat residuals select anchors, and omitted representations are merged into multiplicity-weighted representatives. Results: Corrected simulations with 340 unit-normalized vectors show that the random-walk heat kernel preserves row sums exactly and agrees with its similarity-transformed symmetric implementation to numerical precision. Across six controlled contexts, multiplicity-aware merging preserved approximately 0.99999 attention output cosine fidelity at 8% and 12% retention, whereas residual-only pruning degraded sharply. Anchor rules were effectively tied under this synthetic regime. Conclusions: HKCC is supported as a controlled proof of concept for representation–compression in self-hosted or otherwise modifiable transformer stacks. The current study does not establish savings for commercial text application programming interfaces (APIs), downstream task preservation, or universal superiority of heat residual anchors. The added utility of diffusion, real-model key/value and positional integration, and end-to-end prefill speedups remain unestablished.