Pos1493-HPR Research Transparency and Reproducibility in Rheumatology: A 20-Year Comparison

Background: The current practice of evidence-based medicine stems from high-quality research, ideally free of bias and reproducible by others [1]. Recent reports suggest that there may be a "reproducibility crisis" among publications, in which the majority of studies are unable to be replicated, and are associated with biased or even false result¹. The ramifications could be misinformed medical practices that might alter treatment outcomes. However, the current situation regarding transparent and reproducible practices in rheumatology research has yet to be elucidated. Objectives: This study aims to investigate the transparency and reproducibility in rheumatology research and their changes across 2003 and 2023. Methods: A systematic search of literature was performed on PubMed for all articles published in rheumatology journals that are indexed on MEDLINE and in English. From which, we identified the records in 2003 and 2023, and randomly sampled 100 publications from each year. Two independent investigators reviewed their full texts and extracted the relevant information manually and alternately. Data extracted includes the article's metadata (title, authors, journal of publication and country of origin), its open access status (open access or subscription-based), the funding source statement and the conflict-of-interest statement, if any. For empirical studies (containing original data and/or analysis, excluding case reports/series), their availability statements of protocol, data, materials, analysis script, as well as the statuses of pre-registration, were also reviewed, if any. These characteristics were then compared between publications in 2003 and 2023. Results: A total of 138838 records were identified on PubMed, of which 3129 and 7452 records were published in 2003 and 2023, respectively. Among the 200 randomly sampled articles, 193 (96.5%) were accessible and were further reviewed and analysed. Overall, only around half of the articles were open access (110/193, 57.0%), declared their funding sources (95/193, 49.2%), and disclosed their conflicts of interest (100/193, 51.8%). Significantly more studies in 2023 indicated their funding sources (70.8% vs. 27.8%, p<0.001), and conflicts of interest (88.5% vs. 15.5%, p<0.001), than those published in 2003. Besides, from 2003 to 2023, significant improvements were seen in the specifications of protocol availability (0.0% vs. 9.3%, p=0.027), data availability (1.8% vs. 64.8%, p<0.001), material availability (1.8% vs. 18.5%, p=0.004), and pre-registration status (0.0% vs. 13.0%, p=0.006), but not analysis script availability (0.0% vs. 3.7%, p=0.243) (Table 1). Conclusion: Transparency and reproducibility remain suboptimal in rheumatology research. While significant progress has been made, especially regarding data availability, the overall ratio of studies with available (or at least indicated to be available) protocol, materials, analysis script, and pre-registration remains low even in recent years. Research practises in rheumatology could be further optimised, and more efforts, resources and policies should be put in place. REFERENCES: [1] Baker M. 1,500 scientists lift the lid on reproducibility. Nature 2016; 533:452-4. Acknowledgements: NIL. Disclosure of Interests: None declared.