Towards Sustainable Curation: Evaluation of Cost and Accuracy of AI Tools in Scaling Annotation Tasks in Curation of Biomedical Literature

Puntos clave

• The study aims to evaluate the cost and accuracy of various language models for annotating biomedical literature.
• Compared four language models: GPT 4, Llama 3, Gemma 2, and Mixtral 8x7b.
• Assessed performance in population group curation tasks.
• Evaluated cost-effectiveness in annotation processes.
• Identified variations in performance among the four language models.
• Highlighted potential for cost-saving strategies in annotation tasks.
• Suggested that AI tools could enhance sustainable practices in biomedical curation.