Ask Before You Answer: Distilling Clinical Epistemic Calibration into Model Weights

This preprint introduces a framework for evaluating and instilling the behaviors that distinguish calibrated expert reasoning from confident answer generation in clinical language models. The framework comprises three components: a seven-dimension behavioral decomposition (uncertainty acknowledgment, active inquiry, context-seeking, red-flag identification, scope bounding, hedging quality, and specificity); an asymmetric cross-family grading protocol that prevents filter-grader bias in self-distillation pipelines; and a four-step pipeline that distills chain-of-thought elicited behavior into LoRA adapter weights, removing the need for an inference-time prompt wrapper after training. Applied to MedGemma-27B on HealthBench-Hard across five independent training seeds, the recipe raises active clinical inquiry from 17.5 percent to 45.6 percent (Cohen's d = 0.63), context-seeking from 1.40 to 1.75 on a 0 to 2 scale (d = 0.44), red-flag identification from 1.52 to 1.67 (d = 0.20), and scope bounding from 1.91 to 1.97 (d = 0.22), while preserving aggregate clinical quality (p = 0.57) and eliminating blanket disclaimers. A bootstrap 95 percent confidence interval for the LoRA minus wrapper difference on active inquiry contains zero, indicating that the adapter delivers the wrapper's behavior without the inference-time chain-of-thought cost. The paper additionally surfaces a design constraint relevant to deployment: stacking the inference-time reasoning protocol on the adapted model degrades red-flag identification below the unadapted baseline (d = -0.25), indicating that calibrated behavior interacts non-additively with inference-time scaffolding. This finding is invisible to aggregate quality scoring and only legible through the behavioral decomposition.