Fine-Tuning a Robustly Optimized Encoder Representation from Transformers Model for Domain-Specific Named Entity Recognition in Drilling Reports

Summary The application of language models in petroleum engineering, particularly for the analysis of daily drilling reports (DDRs), has become an area of increasing interest, given the need for automated information extraction. Through this study, we investigated the fine-tuning of a medium-sized language model from the bidirectional encoder representations from transformers (BERT) family (approximately 60–130 million parameters) for named entity recognition (NER) in drilling operations and mud motor reports. A domain-specific corpus was curated using a custom annotation support function to streamline the labeling process and further refined through iterative error analysis. This approach enabled the correction of inconsistencies such as mislabeling of lobe configurations, units, and contextual definitions, ultimately enhancing annotation quality and model performance. Fine-tuning was carried out using low-rank adaptation (LoRA), enabling parameter-efficient training by updating only a small subset of model weights. The fine-tuned model demonstrated strong generalization on a held-out test set, achieving F1-scores exceeding 0.97 across both frequent and infrequent entity types. These findings underscore the importance of high-quality annotations and targeted fine-tuning strategies in achieving reliable domain adaptation. Furthermore, the study highlights that fine-tuned medium-sized models BERT, robustly optimized encoder representation from transformers (RoBERTa), distilled BERT (DistilBERT) can achieve strong performance while requiring significantly fewer computational resources, suggesting their potential suitability for future deployment in offline or edge environments where computational and connectivity constraints limit the use of larger models.