Answers are grounded in this paper and cite their sources.
Introduction: Artificial intelligence (AI) and large language models (LLMs) are rapidly entering resident workflows. However, formal training on appropriate LLM use is limited, and AI outputs may be inaccurate, outdated, or biased. This creates a curricular gap: residents may use AI before receiving guidance on evaluating AI-generated recommendations. Methods: We developed a 30-minute AI critical appraisal workshop for postgraduate year 1 to 3 (PGY-1–PGY-3) internal medicine residents. The workshop replaced 1 evidence-based medicine (EBM) journal club session during ambulatory continuity clinic blocks. Six workshops were delivered from July to December 2025. Learners prompted an LLM using a fictional patient scenario, compared outputs in small groups, and learned a structured appraisal framework. Anonymous surveys were collected immediately before and after the workshop using QR (quick-response) codes. Results: Of 72 eligible residents, 53 completed preworkshop surveys and 44 completed postworkshop surveys. Responses were anonymous and unpaired, so results were analyzed descriptively. Confidence critically appraising AI output increased from 60.4% preworkshop to 79.5% postworkshop. Concern about inaccurate AI predictions increased from 69.8% to 88.6%, and concern about AI bias increased from 77.4% to 88.6%. Belief that AI improves health outcomes remained high: 94.3% and 97.7%. Overall, 90.5% rated the session 4 or 5 out of 5. Discussion: A brief, required workshop teaching residents AI literacy and critical appraisal skills was feasible and well received. Postworkshop respondents reported greater confidence in critically appraising AI output and greater concern about inaccurate predictions and bias than preworkshop respondents. The curriculum is adaptable, requires minimal technology, and can be integrated into existing EBM teaching.
No takes yet. Share an insight, caveat, or question.
Stephenson et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: