Rice's Formula for Event-Driven Networks: Predicting and Training Temporal Sparsity via Differentiable Level-Crossing Rates

Delta networks spend compute only when an activation changes by more than a threshold: eachsupra-threshold change is an event that triggers the corresponding weight-column MACs and memorytraffic. A delta event is, definitionally, a level crossing of the activation time series—yet the deltanetwork literature tunes thresholds by grid search and only ever measures event counts after the fact.We connect this literature to the century-old analytic theory of level crossings. Predict: on three trainednetworks (GRU keyword spotting on Speech Commands, a row-sequential-MNIST GRU, an enwik8 chartransformer), an empirical crossing-rate profile fitted on a calibration split predicts per-channel crossingrates on held-out data to 3.1%–2.9% median relative error, and open-loop send-on-delta simulation oncached traces predicts closed-loop delta-network event rates to within 3.0% (SC2) in the regime wherethe delta approximation preserves accuracy. Rice’s Gaussian formula works where activations are nearGaussian (LayerNorm-fed streams) and fails predictably with kurtosis (GELU outputs, deep GRU states);an iid baseline fails by 5.6×–17.3×. Allocate: inverting the predicted rate curves turns a global eventbudget into per-stream thresholds analytically; on Speech Commands this matches the Pareto frontof a 48-configuration random search at ∼2% of its tuning cost, and realized event rates land within4% of target. Train (negative): fine-tuning against a one-sided differentiable crossing budget makestraces comply (crossings drop as demanded, increment scales shrink 3.9×) but does not improve theaccuracy–events Pareto front over post-hoc thresholding of the unmodified network—and neither doL1-on-deltas or activity-rate controls. The mechanism is an approximate scale-freeness: send-on-deltaevent rates depend on θ/σδ, so smoothness training rescales dynamics without restructuring them, whilefixed-threshold comparisons misleadingly report large savings (36% fewer events at matched absolute θ).Prediction and allocation work; training-for-temporal-sparsity, in these tasks, does not beat the post-hocbaseline it is meant to replace. We release estimators, receipts, and benchmarks.