Clinically grounded evaluation of large language models for traditional Chinese medicine

Medical licensing examination questions provide accessible, standardized benchmarks for evaluating large language models in traditional Chinese medicine. However, examination performance alone cannot establish their ability to support clinical reasoning or improve patient care. This Perspective examines the practical drivers of examination-centered evaluation and four potential sources of bias: the representation of clinical knowledge, coverage of professional competencies, integrity of evaluation data, and compatibility between scoring methods and traditional Chinese medicine practice. These limitations concern the interpretation and scope of benchmark results, rather than the value of standardized knowledge assessment itself. We propose a clinically grounded evaluation framework that combines dynamic clinical scenarios, open-ended generation, multimodal assessment, longitudinal reassessment, and clinical validation. Multi-turn consultations with progressively disclosed information can test reasoning under uncertainty, while prescription generation and multimodal tasks can examine individualized treatment and integration of diagnostic findings. Expert review and automated metrics should assess appropriateness, consistency, and safety, with subsequent evaluation in simulated decision-making and human–AI collaboration. New clinical cases and repeated assessment across model versions should inform updates to evaluation tasks. Recommendation quality, decision efficiency, and patient outcomes should be measured as distinct endpoints. This framework positions examination scores within a broader body of evidence and makes independent clinical validation central to claims of clinical usefulness.