A methodological framework for fractal analysis in monitoring periapical lesions after endodontic retreatment: a comprehensive parameter optimization study

Background This retrospective study aimed to investigate the efficacy of Fractal Analysis (FA) in the quantitative assessment of periapical bone healing after non-surgical endodontic retreatment, and to standardize the key methodological parameters — region of interest (ROI) selection, image processing filters, and box-counting algorithms — that govern its reliability. Methods Ninety-five patients free of systemic conditions affecting bone metabolism who had undergone endodontic retreatment were divided into two groups: a healed group with PAI < 3 ( n = 60) and a non-healed group with PAI ≥ 3 ( n = 35). The non-healed group served as a clinical non-healing reference group to verify whether FA parameters remained stable in the absence of biological change. A total of 11,970 Fractal Dimension (FD) measurements were performed on 190 images (7 ROI types × 3 filters × 3 algorithms = 63 combinations). Seven ROI types (ROI-1 through ROI-7), three Gaussian filters (σ = 4, 12, 32 pixels), and three box-counting algorithms (B64, B32, BoneJ Auto [BA]) were tested. Mixed-effects GLM was used as the primary analysis; paired t-tests with Benjamini–Hochberg FDR correction ( q < 0.05) served as the confirmatory analysis. Results In the healed group, FD increased significantly over one year of follow-up ( p < 0.001, Cohen's d = 1.07, AUC = 0.91 [ROI-1, BA + σ4; 95% CI: 0.85–0.96]). In the non-healed group, no significant FD change was observed for any combination (all p > 0.05, d ≤ 0.07). The highest discriminatory power was achieved with lesion-focused large ROIs, and the BA + σ4 combination yielded the strongest results across all significant regions. Conclusion FA is a powerful quantitative tool complementary to PAI for monitoring endodontic healing. A lesion-focused circular/polygonal ROI (ROI-1/ROI-2), σ = 4 filter, and BoneJ Auto algorithm is recommended as a standardized protocol for 2D FA studies.