Ensemble-Based Multimodal Deep Learning for Precise Skin Cancer Diagnosis: Integrating Clinical Imagery with Patient Metadata

The rising incidence of skin cancer necessitates scalable and accurate diagnostic tools. While dermoscopy-based systems have achieved expert-level performance, clinical smartphone images pose challenges due to variability in lighting, resolution, and artifacts. Recent advances in multimodal deep learning have shown promise, yet most approaches rely on simple feature concatenation or single-model classifiers, limiting their ability to capture complex cross-modal interactions. This study aims to bridge the diagnostic gap in resource-limited settings by developing a robust multimodal framework that synergizes clinical smartphone images with structured patient metadata for automated skin cancer classification. We propose a novel hybrid architecture integrating a Swin Transformer V2 (SwinV2-Tiny) for hierarchical visual feature extraction and a Denoising Autoencoder (DAE) with PCA for robust metadata embedding. These heterogeneous modalities are fused via a Gated Attention Mechanism that dynamically weighs feature importance across streams. Classification is performed by a Heterogeneous Meta-Stack Ensemble comprising CatBoost, LightGBM, XGBoost, and Logistic Regression, designed to maximize calibration and generalization across imbalanced classes. Evaluated on the PAD-UFES-20 dataset (2298 clinical smartphone images, six diagnostic classes), the proposed framework achieves state-of-the-art performance with a macro-averaged F1-score of 0.977, accuracy of 0.978, and an AUC of 0.990. It significantly outperforms unimodal baselines and existing multimodal methods, demonstrating superior sensitivity (0.974) and precision (0.981), particularly for underrepresented malignant classes like Melanoma (F1: 0.995) and Squamous Cell Carcinoma (F1: 0.960). The integration of clinical metadata with advanced visual embeddings via gated attention significantly enhances diagnostic reliability. Comprehensive ablation studies confirm the contribution of each architectural component. This framework offers a viable pathway for deploying high-precision, AI-driven dermatological screening tools on standard smartphone devices.