Enhanced Prostate Tumor Segmentation in MRI Using Hybrid Optimization and Adaptive Thresholding

ABSTRACT Background Accurate segmentation of prostate tumors in magnetic resonance imaging (MRI) is critical for improving diagnostic accuracy and supporting clinical decision making. However, many existing approaches rely on supervised learning methods that require large annotated datasets and substantial computational resources, limiting their clinical applicability. This study aims to develop and evaluate a fully unsupervised framework for prostate tumor segmentation in multiparametric MRI using hybrid optimization and adaptive thresholding techniques. Methods This study proposes an unsupervised prostate tumor segmentation framework based on hybrid optimization and adaptive thresholding. Two metaheuristic optimization algorithms, chaotic particle swarm optimization and forest optimization, were employed to optimize Otsu's variance‐based thresholding and Kapur's entropy‐based thresholding, resulting in four hybrid configurations. The framework was evaluated using multiparametric prostate MRI datasets, including apparent diffusion coefficient, T2‐weighted, and diffusion‐weighted imaging sequences. Segmentation performance was assessed using overlap‐based and classification‐based metrics. Statistical analysis included the computation of descriptive performance measures and confidence intervals to evaluate robustness and consistency across datasets. Results The proposed framework demonstrated reliable and consistent segmentation performance across all MRI modalities. The Otsu‐based hybrid configurations showed superior overlap and classification performance in diffusion‐based imaging, whereas the entropy‐based methods exhibited more conservative behavior on heterogeneous T2‐weighted images. Overall, the optimization‐based approaches achieved high segmentation accuracy and stability without the need for annotated training data. Conclusions The proposed hybrid optimization and thresholding framework provides an effective, fully unsupervised solution for prostate tumor segmentation in multiparametric MRI. Its robustness, computational efficiency, and independence from training data highlight its potential for integration into clinical prostate cancer diagnostic workflows.