1962-P: Evaluation of Large Language Model Performance for Detecting Continuous Glucose Monitoring Patterns against Triple-Labeled Expert Consensus

Introduction and Objective: CGM patterns inform diabetes management, but manual detection is labor-intensive. Whether LLMs can match expert performance and how input modality affects detection remains unexplored. This study evaluates LLM against expert curated labels to determine if LLMs achieve clinically acceptable accuracy and whether input modality significantly impacts performance. Methods: We compared GPT-4o-mini predictions against expert labels for five CGM patterns (afternoon spike, dawn phenomenon, morning hyperglycemia, nocturnal hypoglycemia, overnight compression). Three independent clinical experts (RDN, CDCES, and BC-ADM) reviewed 7-day CGM tracings and labeled patterns. We evaluated three modalities: (1) raw—timestamped individual readings; (2) image—visual time-series plots; and (3) aggregate—3-hour statistical summaries. Ground truth was established via majority vote from 111 triple-labeled instances. We calculated precision, recall, F1 score, and accuracy, and assessed inter-rater reliability via perfect agreement rates. Results: Inter-rater perfect agreement ranged from 0% (overnight compression) to 49% (morning hyperglycemia). Raw data points achieved the highest F1 of 80.7% (recall 96.1%), while image and aggregate achieved 77.0% and 76.9%, respectively. Pattern-specific results revealed modality dependencies: nocturnal hypoglycemia was detected exclusively by raw/text (F1 88.0%), while afternoon spike and morning hyperglycemia performed well across modalities (F1 >85%). Overnight compression failed across all modalities (0% F1). Conclusion: Pattern-specific prompts with strategic modality selection achieved F1 scores of 76.9-80.7% (66% improvement over baseline). Raw/text modality offers practical clinical deployment with high recall, though modality impact varies by pattern. These findings demonstrate that prompt engineering and modality selection are essential for accurate CGM pattern detection. Disclosure J. Quan: Employee; Current; iHealth Labs. W. Hou: Employee; Current; iHealth Labs, Inc. L. Wang: Employee; Current; iHealth Labs, Inc. E. Arredondo: None. N. Ghamrawi: Employee; Current; iHealth Labs, Inc. L.A. Groppo: None.