Generating MeSH terms using large language models: From individual models to ensemble strategies

This study evaluates how effectively large language models (LLMs) can assign Medical Subject Headings (MeSH) to biomedical articles, examining individual models and ensemble strategies. Five LLMs were tested: two biomedical (MMed-Llama-3-8B and Med42-v2-70B), two general-purpose (Mistral-7B and Llama-3-70B), and one commercial (Claude). Open-source models were adapted through Low-Rank Adaptation fine-tuning and retrieval-augmented generation (RAG); Claude through five-shot prompting and RAG. Four ensemble strategies were applied: majority voting, Bayesian fusion, reliability-weighted voting, and cascade. On 1730 target articles, Claude with RAG achieved the highest F1-score (58.4%) through high recall but lower precision, while fine-tuned Llama-70B and Med42-70B showed the opposite pattern. The ensemble strategies compensated for this precision–recall trade-off: majority voting (F1 = 67.8%) and reliability-weighted voting (F1 = 67.6%) achieved balanced performance, exceeding the best-performing Claude by more than nine points. Bayesian fusion achieved the highest recall (82.5%), and cascade achieved the highest precision (82.1%). These precision–recall profiles map onto specific stages of the MeSH indexing workflow: Bayesian fusion for candidate generation, cascade for filtering, and majority voting or reliability-weighted voting for end-to-end indexing. The study therefore gives indexing practitioners a basis for selecting a configuration by task rather than by aggregate accuracy alone.