Fake publications in biomedical science: red-flagging method indicates mass production

Abstract Integrity of academic publishing is increasingly undermined by fake publications massively produced by commercial “editing services” (so-called “paper mills”). These services use AI-supported production techniques at scale and sell fake publications to students, scientists, and physicians under pressure to advance their careers. Because the scale of fake publications in biomedicine is unknown, we developed an easy-to-apply rule to red-flag potentially fake publications and estimate their number. After analyzing questionnaires sent to authors of published papers, we developed simple classification rules and tested them in a 9-step bibliometric analysis in a sample of 17,120 publications listed in PubMed®. We first validated various simple rules and finally applied a multifactorial tallying rule comparing 400 known fakes with 400 random (presumed) non-fakes. This rule was then applied to 1,000 random publications each from 2020 and 2023. The multifactorial tallying rule was the best red-flagging method, with a 94% sensitivity and only a 11.5% false-alarm rate. The rate of red-flagged articles increased during the last decade, reaching an estimated 14.9% in 2020 and 16.3% in 2023. Countries with the highest proportion of read-flagged publications were China, India, Iran, Russia, and Turkey, with China and India the largest absolute contributors globally. Applying Bayes’ rule resulted in an estimate of 5.8% actual fakes in the biomedical literature. Given 1.86 million Scimago-listed biomedical publications in 2023, we estimate the actual number of true fakes at 107.800 articles per year, growing steadily. Scientific publications in biomedicine can be red-flagged as potentially fake using fast-and-frugal classification rules to earmark them for subsequent scrutiny. When applying Bayes´rule, the annual true scale of fake publishing in biomedicine is about 19 times that of the 5.671 biomedicine retractions in 2023. This scale of fraudulent publishing is concerning as it can damage trust in science, endanger public health, and impact economic spending. But fake detection tools can enable retractions of fake publications at scale and help prevent further damage to the permanent scientific record.