MedEdge-Conf: A Lightweight Confidence-Scoring Framework for Edge-Deployed LLMs in IoMT Health Monitoring

Remote patient monitoring systems based on the Internet of Medical Things (IoMT) increasingly use large language models (LLMs) to convert vital signs into readable alerts, but lightweight edge-deployed LLMs may hallucinate, contradict abnormal vitals, or overstate confidence. This study proposes MedEdge-Conf, a lightweight confidence-scoring frame-work combining physiological coherence aligned with the National Early Warning Score 2 (NEWS2), token-level entropy, and semantic stability into a calibrated composite score driving a deliver/review/escalate decision. We evaluate MedEdge-Conf on 249 test scenarios from two open-access PhysioNet resources (CinC2019, MIT-BIH), using three edge-deployed models (Phi-2, TinyLlama, Gemma-2B, all Q5_K_M), with ground truth from two adjudicated human raters per model. Calibrated fusion reduces the false acceptance rate (FAR) of dangerous hallucinations from 100 percent under naive delivery to 26.4 percent for Phi- 2, 17.3 percent for TinyLlama, and 32.1 percent for Gemma- 2B, with AUROC improvement over naive scoring statistically significant in every case (DeLong’s test, p less than 0.03); precision on delivered responses remains low (14.0-22.8 percent), so it filters dangerous delivery rather than guaranteeing reliability. An ablation study shows the signal driving this improvement is model-dependent: physiological coherence alone is strongest for Phi-2, token entropy helps TinyLlama, and semantic stability matters most for Gemma-2B. Against three literature baselines, MedEdge-Conf achieves the highest AUROC and lowest FAR for every model. Real Raspberry Pi 5 measurements across nine model-quantization configurations show TinyLlama is fastest and lightest, while Gemma-2B is markedly slower and heavier despite being the safest raw generator.