Building safer clinical agents: the case for residency-level benchmarks in medical artificial intelligence

Advances in large language models (LLMs) have accelerated medical benchmarking, yet most evaluation of LLMs still relies on exam-style question answering or curated vignettes that test knowledge rather than performance in clinical workflows.1 Such benchmarks only show whether a model has medical knowledge, not whether it can function safely when performing electronic health record (EHR) tasks asked of resident physicians—retrieving patient information from complex records, reconciling conflicting data, tracking changes in patient outcomes over time, and communicating decision-making factors.