Biomedical Named Entity Recognition using Transformer-Based Approaches

Named Entity Recognition (NER) in Biomedical Natural Language Processing (NLP) is a key challenge that is aimed at the automatic identification and classification of biomedical named entities from unstructured biomedical texts, including, but not limited to, chemicals, genes, diseases, organisms, drugs, and anatomical structures. Accurate BioNER enables applications downstream, such as biomedical literature mining, clinical decision support, pharmacovigilance, knowledge graph construction and precision medicine. But the challenges to BioNER are complex biomedical terminology, semantic ambiguity, overlapping entity boundary and strong class imbalance for entity categories. In this thesis, a complete biomedical Named Entity Recognition framework with advanced preprocessing of biomedical data, Biomedical language models, Ensemble learning and Class balancing techniques are proposed to enhance the biomedical Named Entity Recognition accuracy and robustness. A biomedical corpus of 20,000 documents was collected from four sources: PubMed, FDA drug descriptions, USPTO patent abstracts, and ClinicalTrials.gov and annotated with 24 biomedical entity categories. The document cleaning and tokenization were conducted, then documents were normalized, tagged with BIO, validated for entities, deduplicated, and randomly partitioned into three sets: training, validation, and testing. Five pre-trained transformer models were fine-tuned and tested: PubMedBERT, BioClinicalBERT, SciBERT, BlueBERT and RobertaBio. Class imbalance was addressed by using weighted loss, focal loss, and data augmentation methods in model training. In addition, a weighted ensemble framework was proposed to fuse predictions from all transformer models, which leverages their complementary strengths. The system was implemented in an architectural structure that was divided into a RESTful API, a GPU-accelerated inference engine, a model management layer, a caching layer, and a graphical user interface for facilitating the practical biomedical text analysis. Five-fold cross validation and statistical significance testing were also used in the experimental evaluation, along with Precision, Recall, F1-score, Exact Match, Relaxed Match, Macro F1, Micro F1, Weighted F1 and Matthews Correlation Coefficient. The best performance was seen in RobertaBio (Precision = 0.864, Recall = 0.855, F1-score = 0.859), closely followed by PubMedBERT (F1-score = 0.854). On almost every biomedical entity category, the proposed ensemble model achieved higher accuracy than all individual transformers and significant improvements over the difficult, low-frequency entities. The integrated balancing strategy boosted the mean minority-class F1 score from 0.821 to 0.830, which highlights the fact that it managed to reduce the class imbalance without compromising the system performance. Experimental results show that the domain-specific transformer language models are clearly superior to the general scientific language models for biomedical NER, and ensemble learning also improves prediction accuracy, robustness and generalization. The proposed framework is efficient, scalable and reliable for biomedical information extraction and can be utilized for various downstream biomedical applications such as biomedical knowledge discovery, biomedical literature mining, clinical decision support, pharmacovigilance, and intelligent healthcare systems. This research's results help enhance the scope of using transformers in biomedical NLP, offering a pragmatic and efficient platform for scaling up biomedical entity recognition