Extracting and Classifying Drug Discontinuations from Estonian Electronic Health Records: Development and Validation Study (Preprint)

BACKGROUND Drug adherence is crucial for chronic disease management, yet treatment discontinuation remains common due to factors such as side effects, inefficacy, or cost. These reasons are often recorded only in free-text clinical notes, making large-scale analysis difficult. While large language models (LLMs) can interpret such unstructured data more effectively than traditional natural language processing methods, few studies have systematically categorized reasons for discontinuation or identified whether the decision was initiated by the patient or the clinician, especially in low-resource languages like Estonian. OBJECTIVE To assess the ability of LLMs to extract and classify reasons for drug discontinuation and identify who initiated it using Estonian electronic health records, and to characterize the observed discontinuation patterns and initiators for statins and antidiabetic medications. METHODS We combined prescription data with free-text anamneses from a 10% sample of the Estonian population (2012–2019). LLMs (Llama-3.1-70B and GPT-4o) were applied to extract discontinuation phrases and reasons, classify them into a clinician-developed taxonomy, and identify who discontinued the treatment. Performance was evaluated on randomly chosen 100 cases per drug group. RESULTS Extraction yielded 625 antidiabetic drug and 233 statin discontinuation cases. Validation confirmed high accuracy, with 93–98% of extracted phrases and 95–96% of extracted reasons judged correct. Classification of discontinuation reasons achieved weighted F1 scores of 0.808–0.836, while classification of who initiated discontinuation achieved weighted F1 scores of 0.645–0.774. Adverse reactions were the most frequent reason overall, accounting for ~70% of discontinuations for statins and ~44% for antidiabetic drugs. For antidiabetic drugs, treatment inefficacy and contraindications were more common. Patients more often stopped due to adverse reactions or non-medical reasons, while physicians more often initiated discontinuation for contraindications. CONCLUSIONS LLMs can accurately extract and classify medication discontinuation reasons and initiators from Estonian clinical narratives. Both local and proprietary models performed well, enabling scalable analyses that complement structured health records. This demonstrates the potential of LLMs to unlock information from clinical notes, turning this underutilized EHR component into a valuable resource for monitoring treatment patterns and detecting adverse event signals.