What question did this study set out to answer?

February 6, 20260 citationsOpen Access

Improving the Reporting Quality of Studies on Information Extraction From Clinical Texts: Protocol for the Development of a Consensus-Based Reporting Guideline

DRDaniel ReichenpfaderUniversity of Geneva HMHenning MüllerUniversity of Geneva KDKerstin Denecke

Key Points

The aim is to create a consensus-based reporting guideline for studies on clinical information extraction.
Development follows established guideline methodology.
Encompasses a 3-round electronic Delphi study with domain experts.
Final in-person consensus meeting at the MEDINFO 2025 conference.
Items rated on a 10-point scale for relevance and maturity.
Drafted a preliminary set of 28 reporting items from review and frameworks.
Will result in a validated reporting guideline for clinical information extraction studies.
Draft guidelines include 5 dimensions: information model, architecture, data, annotation, and outcome.
Designed to be technology-agnostic across diverse information extraction approaches.

Abstract

Background: Information extraction (IE) from clinical texts is increasingly important in health care; yet, reporting practices remain inconsistent. Existing guidelines do not fully address the unique challenges of IE studies. IE methods vary widely in their design, ranging from rule-based systems to advanced large language models, contributing to heterogeneity in reporting. While several reporting frameworks exist for applications of artificial intelligence in health care, they primarily focus on prediction modeling or clinical trials and associated protocols rather than text-based IE. Objective: This study aims to develop the Clinical Information Extraction (CINEX) guideline, a consensus-based reporting guideline for studies on clinical IE. Methods: The CINEX guideline is developed following an established guideline methodology, including a 3-round electronic Delphi (eDelphi) study with domain experts and a final in-person consensus meeting. The eDelphi process includes feedback loops and predefined consensus thresholds, with items rated on a 10-point scale for both relevance and maturity. The final consensus meeting is held as a hybrid workshop at the MEDINFO 2025 conference and focuses on finalizing the items that reached consensus. Results: Our results will provide a validated reporting guideline for studies on clinical IE. A preliminary set of 28 reporting items was drafted from a scoping review and existing frameworks. The draft guidelines include 5 key dimensions: information model, architecture, data, annotation, and outcome. This draft guideline will be refined through the eDelphi process. It is designed to be technology-agnostic and applicable across diverse IE approaches, including not only large language models but also traditional machine learning methods and rule-based and hybrid systems. Conclusions: The CINEX guideline provides structured, expert-validated guidance for reporting clinical IE studies, improving transparency, reproducibility, and comparability. The final guideline will be disseminated alongside an explanatory document to support adoption and implementation.

AIに質問

Bookmark

View Full Paper