Machine Learning Methods of Symptom Extraction from Medical Notes

Medical notes contain valuable information about patient conditions, treatments, and progress. Extracting symptoms from these unstructured notes is crucial for clinical research, population health analysis, and decision support systems. Traditional manual methods are time-consuming, but recent advances in natural language processing (NLP) and machine learning offer automated solutions. This article presents a novel approach that combines NLP techniques, such as conditional random fields (CRF) and transformer-based architectures. The proposed method demonstrates effective symptom extraction from medical notes, overcoming challenges such as varied terminologies and linguistic nuances. The study utilizes a dataset of Russian medical records, transforming it into a tabular format for training and employing unique tokenization algorithms for different models. Among the evaluated models, RuBERT achieved the highest accuracy of 91%, indicating its strong performance on the test dataset. SBERT exhibited the highest precision and F1 score, suggesting its effectiveness in accurately identifying specific sequence labels.