Enhanced multiclass classification of AML-related cells using YOLOv12, Inception-ResNet-v2, and ResNet50 transfer learning models

Introduction Acute Myeloid Leukemia (AML) is an aggressive hematological malignancy whose diagnosis relies heavily on the accurate identification of abnormal white blood cells. However, multiclass classification of AML-related cell types remains challenging due to the high visual similarity among different leukocyte categories. Methods This study investigates the classification of multiclass AML-related cells using three deep learning models: ResNet50, Inception-ResNet-v2, and YOLOv12. Two segmentation strategies based on cell-level and nucleus-level representations were evaluated using Hue-channel extraction and Otsu thresholding to enhance morphological feature extraction prior to classification. Results Experimental results show that YOLOv12 combined with Otsu-based cell segmentation achieved the highest test accuracy of 99.3%. Inception-ResNet-v2 achieved a test accuracy of 97.6% using Hue-based cell segmentation, while ResNet50 achieved 93.87% using nucleus-based Otsu segmentation. The findings demonstrate that preserving complete cellular morphology significantly improves classification performance. Conclusion The results highlight the effectiveness of combining segmentation techniques with modern transfer learning architectures for AML-related cell classification. In particular, Otsu-based cell segmentation and YOLOv12 provided the most discriminative representation of cellular morphology, enabling highly accurate classification of challenging cell types such as basophils and monocytes.