A deep learning approach based on YOLO v11 for automatic detection of jaw cysts

OBJECTIVE: Jaw cysts are frequent radiolucent lesions in dentistry that can present diagnostic difficulties due to their similar radiographic appearance. This study aimed to develop an AI-based detection and classification system for jaw cysts using the YOLO v11 deep learning model on panoramic radiographs. MATERIALS AND METHODS: A total of 311 panoramic images (211 cystic, 100 normal) were labeled and augmented. The YOLO v11 Small model was trained in both multi-class (distinguishing between dentigerous cysts DCs, odontogenic keratocysts OKCs, and radicular cysts RCs) and single-class configurations (detecting cysts without type differentiation). Performance metrics included precision, recall, F1 score, and mean average precision (mAP). RESULTS: In the multi-class model, the system achieved an mAP of 86%, with precision of 84%, recall of 82%, and F1 score of 83%. Class-wise mean accuracies were 91% for DCs, 85% for OKCs, and 82% for RCs. The single-class model showed slightly lower performance with an mAP of 84% and F1 score of 81%. CONCLUSION: YOLO v11 demonstrated high accuracy in detecting jaw cysts, indicating its potential to support dental diagnostics. Further validation on larger and balanced datasets is recommended to enhance generalizability.