PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
June 1, 2026Psychological Bulletin0 citations

Artificial intelligence as a partner in meta-analysis—Research agenda, user recommendations, and speed–accuracy tradeoffs: Commentary on Jansen et al. (2025).

View Full Paper
BMBrooke N. Macnamara

Key Points

  • The commentary aims to explore the potential of artificial intelligence in enhancing meta-analyses through improved data extraction.
  • Reviewed performance of eight large language models in data extraction from thousands of studies.
  • Identified inaccuracies and proposed methods for using AI in systematic reviews.
  • Recommended a research agenda addressing AI-induced bias and coding independence.
  • AI demonstrated impressive efficiency, completing tasks in under an hour that would take humans over 6,500 hours.
  • Accuracy varied significantly based on the type of variable extracted.
  • Proposed new methods to enhance AI's role as a partner in data extraction for meta-analyses.

Abstract

Using a metascience framework for improving meta-analyses, Jansen et al. (2025) tested the accuracy and efficiency of data extraction from primary studies used in meta-analyses with a range of large language models. Efficiency was impressive: Across thousands of studies and hundreds of variables, eight large language models took less than an hour combined to extract hundreds of thousands of data points-work estimated to take a human coder >6,500 hr. Nevertheless, accuracy was inconsistent, ranging from high to low depending on the variable. From these results, Jansen et al. recommended (a) a research agenda for investigating the use of artificial intelligence (AI) for data extraction and (b) methods for using AI as a partner for data extraction when conducting systematic reviews. This commentary expands on recommendations for the research agenda, such as investigating AI-induced bias, the illusion of exploratory depth, and using AI to extract study quality data. This commentary also offers further considerations regarding using AI as a meta-analysis partner, such as how iterative prompts might reduce coding independence. Finally, the commentary discusses speed-accuracy tradeoffs in meta-analyses. (PsycInfo Database Record (c) 2026 APA, all rights reserved).

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Brooke N. Macnamara (2026) studied this question.

synapsesocial.com/papers/6a1d20f302fbce91306372e2https://doi.org/10.1037/bul0000519
Ask AI
Helpful
Bookmark
Share
View Full Paper

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

  1. 1Unmasking bias in the evidence ecosystem: a panoramic analysis of 311,751 meta-analyses using an artificial intelligence agent-based approach2026
  2. 2The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis2025
  3. 3Can AI Lend a Hand? Integrating Artificial Intelligence into Education Systematic Reviews and Meta-Analyses2025
  4. 4Incorporating Artificial Intelligence in Medical Research Publications: Boon or Bane?2026
  5. 5Artificial intelligence–assisted statistical analysis and statistical review: evidence (2023–2025) and implications for internal medicine2026 · 2 citations