What question did this study set out to answer?

This research aims to develop and validate a Fisher Information based framework for governing hallucinations in autoregressive large language models.

June 17, 2026Open Access

Fisher Information Spectral Sensitivity as a Runtime Hallucination Governance Signal: Multi-Phase Empirical Validation on Apple M5

Key Points

This research aims to develop and validate a Fisher Information based framework for governing hallucinations in autoregressive large language models.
Multi-phase empirical validation on Apple M5 Silicon, N=206 evaluations in Phase 1.
Surface-proxy token-budget gating and spectral FIM monitoring across multiple models.
Correlational and calibration analysis using statistical methods such as Pearson and Spearman coefficients.
Phase 1 found a Pearson r=0.806 for proxy-signal consistency; token savings of 34.0-34.6% for various models; real battery-drain energy saving of +8.1%.
Phase 2 showed a signal-output correlation of r=0.962, with an 85.6% detection rate for early warning of hallucinations.
Causal intervention displayed a significant effect with Cohen’s d=1.95 for post-onset tokens.

Abstract

We present the Kerimov–Alekberli (K–A) framework, a runtime hallucination governance architecture for autoregressive large language models (LLMs) grounded in FisherˆInformation geometry. The framework’s core signal, ωmax(Ft) — the dominant eigenvalue of the per-token empirical Fisher Information Matrix (FIM) — measures the curvature of the model’s log-likelihood surface and flags distributional departure preceding hallucination. The paper reports a two-phase empirical programme on Apple M5 Silicon (32 GB unified memory). Phase 1 (surface-proxy token-budget gating, Ollama v0.23.2, five models, N =206 valid evaluations) establishes: Pearson r=0.806, Spearman ε=0.728 (n=28, p<0.001) for proxy-signal consistency; per-model bootstrap 95% CI token savings of 34.0±4.0% 31.9, 36.3 (llama3.2, n=12), 34.6±2.9% 32.5, 36.6 (gemma3:latest, n=6), 30.8±5.7% 27.4, 34.8 (gemma3:27b, n=8); real battery-drain energy saving of +8.1% overall (+43.1% for factual queries, →0.6% for reasoning); and a critical calibration finding: the empirical FIM threshold at the 95th percentile is ϑcal=15.9 (replicated 17.7 in a second run), versus the prior theoretical estimate of ϑth=0.065 — a 244-fold drift that establishes the necessity of hardware-grounded calibration. Phase 2 (spectral FIM monitoring, Ollama v0.30.7, 12 models, top-K=20 analytic ωmax approximation) yields four pre- registered experimental results. E7 (stability gate): 10/12 models pass CV<0.20; 11/12 after think:false correction. E1 (signal–output correlation): r=0.962 (p=1.4↑10→28) for qwen3:32b (think:false); median r=0.479 across nine non-thinking models. E6 (early warning): median 27-token lead-time (IQR 12–39), detection rate 85.6%, t=33.2, p=2.2↑10→114 (n=1,080 trajectories). E3 (causal intervention):→66% post-onset tokens (Cohen’s d=1.95, p=3.3↑10→132, dose–response confirmed, !H n.s.). Phase 3 (full-vocabulary exact FIM via mlx-lm, 7 models, vocab 100K–262K) is ongoing. Open limitations are stated explicitly: E1 model-dependence (median r=0.479), waste-oracle circularity, Phase 1 hallucination-reduction null result (0.0 pp), and quantisation scope (all results: 4-bit Q4 K M).

Read Full Paperexternally

Ask AI

Helpful

Bookmark

View Full Paper