Answers are grounded in this paper and cite their sources.
Background Advances in clinical NLP depend on high-quality datasets that reflect the complexity of real-world documentation. In response, a fine-grained, relation-annotated corpus for urinary stone diagnostics is introduced. The corpus addresses a gap in German medical NLP resources and provides a reference dataset for developing and evaluating clinical information extraction models. Methods We extracted and de-identified 262 radiology reports and 251 corresponding short findings concerning kidney and ureter stones from the hospital information system of the University Medical Center Freiburg. Using a detailed annotation schema comprising nine entity types and five relation types, three medical doctoral students annotated the corpus in Label Studio, followed by expert adjudication to establish a gold standard. For evaluation, we fine-tuned several German general-domain and medical language models for named entity recognition (NER) and relation extraction (RE) using a fixed train–development–test split, a unified Hugging Face pipeline, and NNI-based hyperparameter tuning. Results The resulting corpus comprises 513 radiology documents with 3,223 annotated entities, dominated by stone- and anatomy-related categories. Inter-annotator agreement was consistently high for both entity and relation annotations (NER F1 = 82.8%; RE F1 = 81.8%), supporting the reliability of the annotation schema. The evaluated models achieved macro F1-scores of up to 89.45% for NER and 99.24% for RE. While NER performance varied across models and fine-grained entity categories, RE performance was consistently high across all evaluated encoders. Conclusion RECUR establishes a fine-grained reference corpus and evaluation setting for automated information extraction from German radiology reports. Its adjudicated annotations, strong inter-annotator agreement, and comprehensive encoder baselines support the systematic evaluation of clinical NER and RE methods.
No takes yet. Share an insight, caveat, or question.
Schmitt et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: