Tiny Recursive Speculative Decoding: Weight-Tied Multi-Step Latent Recursion in Shared-KV Multi-Token Prediction (MTP-TRM) and Non-Causal Block-Diffusion Draft Models (DSpark-TRM) for Quantized GGUF Foundation Models

Speculative decoding accelerates autoregressive large language model (LLM) inference without sacrificing output quality by generating candidate token sequences with a lightweight draft mechanism and verifying them in parallel with the target model. In production C++/CUDA engines (llama.cpp / llama-server), two distinct speculative architectures have emerged for Gemma-4-E4B-it (L = 42, D = 2,560):(1) External Non-Causal Block-Diffusion Draft Models (DSpark / arch="dflash"), which denoise blocks of B = 8 future tokens simultaneously via bidirectional attention and second-order Markov links, and(2) Internal Shared-KV Multi-Token Prediction Heads (MTP / gemma4-assistant / --spec-type draft-mtp), which recursively apply a compact 4-layer cross-attention head (94.08 MB) directly over the target model's pre-computed Layer 22 Sliding-Window and Layer 23 Global Full-Attention KV caches. Despite their structural elegance, both paradigms suffer from severe Multi-Step Horizon Collapse: in DSpark, positions 1..7 are initialized as identical <mask> tokens (ID 4), causing a single bidirectional pass to guess adjacent tokens independently without mutual grammatical consistency (19.72% acceptance, 68.88 tok/s); in MTP, standard single-step (k = 1) teacher-forced training leaves the recurrent latent bridge zₜ₊ₖ₋₁ -> zₜ₊ₖ uncalibrated against self-generated state drift at deeper speculative steps (k >= 2). Inspired by Tiny Recursive Models (TRM; "Less is More: Recursive Reasoning with Tiny Networks", Jolicoeur-Martineau, arXiv:2510.04871), we introduce Tiny Recursive Speculative Decoding, unifying sub-7M-parameter weight-tied latent recursion, half-detached gradient feedback, and multi-step deep supervision across both speculative paradigms under a strict <= 2-epoch anti-repetitive-bake constraint: 1. Non-Causal Block Diffusion (DSpark v10-TRM-LoRA, 6.62M trainable parameters):We expand the draft vocabulary from 32,000 to 65,536 tokens (t2d_64k, incorporating all 25,822 Japanese, mathematical, and symbolic tokens via target-head phase-locking and normalized Tikhonov Markov projection) and decompose the single non-causal denoiser blk.3 into a K = 2 Runge-Kutta 1/2-step weight-tied recursion (blk.3 === blk.4). Across 1,600 unseen validation blocks, DSpark v10-TRM-LoRA elevates mean block accuracy from 27.70% to 39.84% (+12.14%), pos_0 accuracy from 57.56% to 74.31% (+16.75%), and Expected Acceptance Length (EAL) from 2.786 to 3.491 (+0.705 tokens). On live llama-server.exe, it accelerates generation from 68.88 tok/s (19.72% acceptance) to 88.15 tok/s (1.230x vs. AR, 36.38% acceptance in Folded-4L mode; 53.43% acceptance in Unrolled-5L mode, reaching 77.2% on Python and 79.5% on Math CoT). 2. Shared-KV Multi-Token Prediction (MTP v10-TRM-LoRA, 1.74M trainable parameters):We unroll the 4-layer gemma4-assistant head across K = 3 recursive steps (k = 1, 2, 3) on exact on-policy features extracted exclusively from Hermetic-Airlock v8 (gemma-4-E4B-it-100B-hermetic-airlock-v8-Q4_K_M.gguf), enforcing explicit lower-triangular boolean SDPA masks, runtime RoPE position-shift randomization, 1D RMSNorm/scale freezing, and Elastic Weight Consolidation (EWC) Frobenius trust-region anchoring. Splice-quantized in-place into Q8_0 (94.08 MB), MTP v10-TRM-LoRA improves unseen validation accuracy across all steps k in {1, 2, 3} and achieves 123.91 tok/s (1.729x speedup over the 71.67 tok/s AR baseline, +55.03 tok/s faster than legacy DSpark v9, and +35.76 tok/s faster than DSpark v10-TRM) with 8/8 (100.0%) bit-exact greedy lossless equivalence, setting new records in Japanese (93.07 tok/s), Python (140.05 tok/s, 67.5% acceptance), Math CoT (153.94 tok/s at n_max = 4), and mean accepted length (2.760 at n_max = 3; 3.123 at n_max = 4).