Answers are grounded in this paper and cite their sources.
Large language models increasingly answer gynecologic-oncology questions with citations and management recommendations. In a preregistered 308-item benchmark (252 confirmatory) administered to GPT-5.5, Claude Opus 4.8, and Gemini 3.1 Pro, we tested whether cited papers support the claims they accompany, whether unsupported citations track the volume of indexed literature, and whether otherwise-invariant recommendations change with patient nationality. Citations were verified for existence, record identity, and claim support by an automated judge validated against two blinded gynecologic oncologists and a second judge model. All three models answered ancestry-specific questions correctly (24 of 24). Strict fabrication (non-existent or misattributed citations) was unrelated to literature coverage (β = −0.19, p = 0.22), whereas citation of real, on-topic papers lacking the claimed fact rose steeply as coverage fell (β = −1.12, p = 2 × 10⁻⁸; failure rates 0.67, 0.60, and 0.26 across tertiles), with a similar gradient in the 48 exploration-only items. Nationality-dependent violations of invariant management reached 0.11–0.67 across models when the care setting was unstated versus 0.00–0.13 when a fully resourced setting was stated. Legitimate population differences were preserved. Evaluations should score claim-level citation support and specify the care setting; clinicians should verify that cited sources support the specific claim.
No takes yet. Share an insight, caveat, or question.
Kim et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: