MTDF-Net: A multimodal transformer-based data fusion framework for prostate cancer detection with comprehensive evaluation of performance, generalization, and interpretability

: Multimodal biomarkers such as MRI imaging, histopathology and clinical-risk descriptors are increasingly becoming essential for prostate cancer diagnosis. These modalities are not usually combined into coherent, comprehensible systems in current systems. We introduce MTDF-Net, a multimodal Transformer-based data fusion method by combining modality-specific encoders, cross-modal fusion mechanisms, Particle Swarm Optimization-guided feature refinement, and dual-diagnostic heads for cancer detection and Gleason-risk prediction. In addition to diagnostic accuracy, MTDF-Net features high computational efficiency (O(n log n) complexity, 67% memory reduction), generalizability of the model (2,847 parameters with minimal variance), uncertainty quantification (94.6% calibration coverage), and a high level of clinical interpretability, specifically through Attention Maps and feature analysis with SHAP. The accuracy, F1-score, and AUC from the evaluation on the PI-CAI, PROSTATEx, and TCGA-PRAD datasets were 96.8%, 95.9%, and 0.978, respectively. MTDF-Net achieved the highest accuracy, AUC, and the lowest processing time among different methods, including baseline methods (PI-RADS, U-Net, ResNet50, DenseNet121, Swin-UNet, iPCa-Former, PSO-GPC). The results show that MTDF-Net is a highly reliable prostate cancer diagnosis tool, maintains computational efficiency, and has clinical interpretability.