Answers are grounded in this paper and cite their sources.
Purpose: Large Language Models have captured the research environment with their potential to aid in understanding a multitude of natural language processing tasks. Often they require fine-tuning with corpora of relevant information. Electronic health records have an abundance of free-text that is dense with information and could be used for this purpose. This work aims to create a gold-standard annotated dataset with an associated set of guidelines for the annotation process. Methods: Using a dataset of 1500 electronic health records from epilepsy clinics, we created a set of annotation guidelines and a team of data scientists and clinicians annotated these accordingly. Results: We find an overall inter-annotator agreement of 0.769, as measured with Krippendorff's Alpha, and showed that the most annotated and agreed upon annotations are those related to anti-seizure medications and diagnoses of epilepsy and other comorbidities. Conclusion: This work provides a valuable resource for the development and evaluation of NLP systems for epilepsy and electronic health records.
No takes yet. Share an insight, caveat, or question.
Davies et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: