Visualizing Image Segmentation Network Behavior Through the Lens of Scale Space Analysis

Deep neural networks are widely used for image segmentation, also in sensitive applications such as medical imaging or autonomous driving. However, few explainable AI methods are available that help developers understand such networks beyond classification. We demonstrate that studying how segmentation networks behave in scale space, i.e., when increasingly blurring their input, conveys relevant insight into whether detections require features such as sharp edges or high‐frequency textures, and what priors have been learned implicitly. We introduce a visual analytics framework that supports a systematic qualitative and quantitative investigation of how segmentations evolve across scale space. In particular, summary visualizations for individual images support the formation of hypotheses that are subsequently formalized via customizable feature representations and evaluated on larger sets of test images using interactive embeddings and quantitative plots. On a cardiac MRI dataset, we demonstrate that this framework reveals important differences between a convolutional and a transformer‐based neural network that, at first glance, appear to produce similar results. We confirm the practical relevance of those insights on out‐of‐distribution images from an MR scanner from which no images were included in the training data. Beyond this primary analysis, we provide extended results on the Adverse Conditions Dataset with Correspondences (ACDC) for semantic driving scene understanding.