Why Open Large Language Models Matter in Rheumatology Research: a Scoping Review Objectives: To systematically identify and characterize original rheumatology research using large language models (LLMs), quantify reliance on closed versus open models, and assess reporting practices relevant to reproducibility (model versions, prompts, inference settings, and availability of code/data), in order to propose practical recommendations to strengthen transparency. Methods: We searched PubMed/MEDLINE for English-language peer-reviewed original research published from 1 November 2022 to 23 January 2026. Two reviewers independently screened title/abstract and full texts, with a third resolving disagreements. Data were extracted using an LLM-assisted workflow and then independently verified against source articles by the authors. Extracted items included study characteristics, model families and openness, access mode, versioning/timing, and transparency indicators (prompts, code, data). Results: Of 185 screened records, 63 studies were included. Most were research (n = 33/63; 52.38%) or education-focused (n = 24/63; 38.10%). Studies predominantly used closed-source LLMs (n = 50/63; 79.37%), with limited exclusive use of open-weight models (n = 4/63; 6.35%) and some hybrid use (n = 9/63; 14.29%). OpenAI models were most common (n = 55/63; 87.30%). Reporting was heterogeneous: interaction language was often not reported (n = 42/63; 66.67%); access mode was reported in (n = 19/63; 30.16%); output generation date in (n = 33/63; 52.38%). Prompts were shared in (n = 48/63; 76.19%), but code was publicly available in (n = 4/63; 6.35%), and data in (n = 13/63; 20.63%). Conclusions: LLM-based rheumatology research largely depends on closed models with inconsistent reporting of reproducibility-critical details and minimal code sharing. A concise, rheumatology-oriented checklist is proposed to standardize reporting and improve auditability, replication, and cross-study comparability.
García et al. (2026) studied this question.