Answers are grounded in this paper and cite their sources.
Named entity annotation in clinical text is a costly and time-intensive task, particularly for low-resource languages such as Spanish. This article introduces CLINFEM, an XML-annotated corpus comprising 1,200 Spanish medical reports from two regions of Spain. CLINFEM was created using a thorough human-in-the-loop annotation process combining large language model pre-annotation and systematic expert review. It is imperative to acknowledge that each of the 1,200 medical reports underwent meticulous review and correction by a trained human expert. Large language model pre-annotation was employed solely for the purpose of saving time and was never intended to serve as a substitute for manual annotation. Consequently, the entire CLINFEM corpus is a human-validated, gold-standard resource. GPT-4o and Gemini 1.5 Pro were evaluated as pre-annotators using a custom scheme comprising eight clinical entity types and one relational category. GPT-4o achieved a Cohen's κ of 0.73 against human annotation, while Gemini 1.5 Pro obtained κ = 0.41. Human expert review yielded an overall inter-annotator agreement of κ = 0.869 (F1 = 0.793), with a gold-standard subset agreement of κ = 0.9894. This approach reduced the overall annotation effort by around 74% compared to a fully manual approach (81% for the bulk phase alone), saving over 1,000 hours. Error rates varied by medical speciality, with oncology requiring the highest correction rate (94.3% of reports). These results demonstrate that using large language models to assist annotation is a viable and efficient strategy for constructing clinical corpora in Spanish, while also highlighting the continued necessity of human oversight.
No takes yet. Share an insight, caveat, or question.
Sanchez et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: