Large language models as judges for clinical generative AI evaluation

Scalable evaluation of free-form clinical generative AI outputs remains a major barrier to safe deployment. While LLM-as-a-Judge frameworks offer a potential alternative to manual review, most evidence comes from benchmarks, documentation, or other constrained settings. We evaluated agreement between clinicians and LLM judges using 118 clinician-authored, protocol-grounded clinical queries answered by PANDA, a retrieval-augmented clinical decision-support AI system deployed in a tertiary pediatric hospital. Five physicians and seven LLMs graded all query–response pairs across seven clinically selected rubrics, generating 4130 physician ratings, 17,346 ordinal and 8496 binary LLM ratings. LLM judges showed high concordance on text-comparison rubrics such as faithfulness and completeness, but weaker agreement for interpretive dimensions like reasoning and safety. Repeated runs and multi-model panels did not systematically improve alignment beyond the best single model, while binary pass/fail evaluation exposed misalignments not apparent on ordinal scales. These findings support cautious, rubric-aware use of LLM judges as a scalable evaluation layer for clinical AI.