Observational audit evaluates quantization inconsistencies in clinical language models, suggesting readiness for deployment.
Pre-registered methodology for a within-subject paired observational audit of quantization-induced inconsistency in three sub-4B clinical small language models (Qwen-MediCare-BD 3.1B, Gemma-3-4B-it, Phi-4-mini-instruct 3.8B) across four GGUF K-quant levels (Q6_K, Q5_K_M, Q4_K_M, Q3_K_M) on 100 stratified ACI-Bench encounters with 20 seeds per condition (24,000 generations, 228,000 pairwise metric evaluations). Reference-free metric pipeline: BERTScore-F1, ROUGE-L, MEDCON-F1, numerical Jaccard with paired extraction-volume control, and bidirectional NLI contradiction (three aggregations). Statistical analysis: Friedman omnibus, Wilcoxon-Pratt pairwise, BCa bootstrap with tie-corrected z0, Holm-Bonferroni correction, simulation-based power analysis (MDE δ ≥ 0.246 at 80% power). Three-judge LLM convergent validity panel (Claude Opus 4.7, Gemini 3.1 Pro, DeepSeek Expert Mode) on two parallel tracks. Dual-criterion deployment-readiness matrix
No takes yet. Share an insight, caveat, or question.
Shanto et al. (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: