Large language model-based evaluation of the impact of gender in medical research

Objective: Gender disparities in academic medicine have been previously reported, but prior analyses have relied on either manual labor or fixed databases of name-gender pairs that fail to generalize across different populations and cultures. The objective of this work is to evaluate the utility of large language models (LLMs) as a potential tool to facilitate systematic bibliometric analysis of academic research trends. Methods: We introduce an LLM-based pipeline that aggregates gender labels from multiple LLM instances to predict the genders of manuscript authors based on their first names. Results: Our proposed method outperforms alternative algorithms relying on lookup from finite databases of name-gender pairs, while also offering the scalability to tens of millions of authors that is unfeasible with other manual, human-based methods alone. Discussion and Conclusion: Our results suggest that LLMs can be a powerful tool to scalably track gender-based trends in academic medical research.