PRISM: a clinically interpretable stepwise framework for multimodal skin cancer diagnosis

Skin cancer is the most common type of cancer worldwide, placing a substantial burden on healthcare systems. While numerous computer-assisted diagnostic systems have been developed, most offer limited transparency, making it difficult for healthcare professionals to understand how individual pieces of clinical information influence the diagnostic outcome. In this work, we propose PRISM: Probabilistic Reasoning Interpretable Stepwise Model, an interpretable multimodal skin cancer classification framework that integrates image-based predictions with clinical data using a Bayesian framework. Our approach allows the model to be evaluated with incrementally available metadata, enabling clinicians to interpret how each clinical feature contributes to the diagnostic decision. To address the compounding overconfidence inherent in sequential Bayesian updating, we implement a Stepwise Calibration protocol that dynamically scales with the volume of available evidence, ensuring statistically reliable confidence estimates at every intermediate diagnostic step. We validated the framework with four competitive vision backbones on three datasets. Compared to state-of-the-art attention-based methods, our framework achieves competitive or superior performance, reaching a peak balanced accuracy of 77.2 ± 3.6% on the PAD-UFES-20 dataset. Finally, qualitative case studies indicate that PRISM creates a reasoning process that is consistent with clinical logic, suggesting it is a viable alternative for clinical decision support. An interactive demo is available at https://huggingface.co/spaces/pedrobouzon/prism.