Natural language processing (NLP) to identify neoplastic fever in the electronic medical record (EMR).

e13679 Background: Neoplastic fever (also known as “tumor fever”; TF), is a rare but challenging condition that can be difficult to distinguish from infectious or treatment-related causes of fever. TF can lead to potentially unnecessary diagnostic testing and antibiotic use, as well as physical and psychological distress for patients. While treating the underlying cancer is the gold standard, this may not be successful or sufficient. Further, the inability to prospectively identify individuals with TF limits research to improve prevalence estimates, non-cancer directed therapy and patient quality of life. Therefore, our goal was to develop and validate an NLP algorithm to identify TF from the clinical notes of a retrospective cohort of adult cancer patients. Methods: Research was approved by the Fred Hutchinson Cancer Center Institutional Review Board. Unstructured clinical notes were extracted from the EMR among patients in the Adult Oncology Program with both ICD10 cancer-related diagnoses with at least two outpatient office visits within a 6-month period between January 1, 2018 and December 31, 2024. The study team developed the annotation schema iteratively with clinical and NLP experts for TF and related clinical entities. Annotations occurred over multiple rounds using the MedTator software. Inter-annotator agreement was assessed using F1 scoring and Cohen’s kappa coefficient, with acceptable inter-annotator agreement (AIAA) defined as F1 score of ≥.8. Results: 354,245 clinical notes for 26,774 individuals were identified during the study period. Of these, 117,545 (~33.18%) notes had a term related to TF (e.g. "tumor fever", "neoplastic fever", "fever", "sweating"). 341 charts were annotated from 89 patients for the final gold standard corpus. AIAA was achieved with overall F1 score of .83 (see Table 1) across all entity and relation types. Training and evaluation of NLP models is ongoing. Full results will be presented. Conclusions: Our results suggest NLP can be used to identify true cases of tumor fever in the EMR. To our knowledge, this is the first gold standard annotated corpus specifically designed for tumor fever detection in clinical notes. This resource addresses a critical gap in oncology-focused NLP, providing a foundation for developing and evaluating automated methods to identify tumor fever from EHR data and supporting future research aimed at improving diagnostic accuracy and treatment of tumor fever. Inter-annotator agreement, precision and recall for tumor fever-related annotation labels. Annotation Label F1 Precision Recall True Positive False Positive False Negative OVERALL 0.8263 0.7565 0.9103 2181 702 215 FEVER 0.9433 0.9252 0.9620 507 41 20 FEVER SYMPTOMS 0.9236 0.8788 0.9732 145 20 4 NEOPLASM 0.7874 0.6889 0.9186 722 326 64 CANCER SIGN 0.7457 0.6643 0.8498 566 286 100 NEGATION 0.8078 0.8240 0.7923 103 22 27 TUMOR FEVER 0.9753 0.9517 1.0000 138 7 0