Since different imaging modalities provide complementary tissue structure and lesion information, multi-modal medical image semantic segmentation can significantly improve the ability to identify and locate lesion regions by fusing multi-source image features. However, the existing methods usually model the segmentation task as a pixel-level independent classification problem, ignoring the topological relationship among pixels, and it is difficult to effectively fuse the cross-modal spatial and frequency domain features from the perspective of topological relationship, resulting in blurred lesion boundaries and incomplete lesion contour structure expression. To solve the above problems, this paper proposes a topological spatial–frequency interaction network for cross-modal medical image segmentation. By explicitly modeling the connectivity relationship among pixels and combining the spatial domain and frequency domain information, the cross-modal connectivity topological features are enhanced. Specifically, this paper constructs a three-encoder single-decoder multi-modal segmentation framework based on connected graph modeling, and introduces three types of masks such as connected region, lesion contour and lesion edge for joint supervision to enhance the ability of the model to perceive the edge of the lesion and the contour of the lesion. In the network structure, a cross-modal "frequency-channel, spatial-location" topology feature compensation module is designed in the skip connection stage to enhance the topology information transmission between the encoder and the decoder. In the bottleneck layer, a cross-modal "spatial-channel, frequency-location" topological feature enhancement module is proposed, which decomposes the cross-modal connectivity topological features at high and low frequencies through discrete wavelet transform, and combines the self-attention mechanism to realize the interactive fusion of spatial domain and frequency domain information, so as to improve the expression ability of deep cross-modal topological features. Compared with the baseline model, on the lung tumor dataset, mIoU, Dice, VOE, RVD and Recall are increased by 3.03%, 2.34%, 4.76%, 3.68% and 2.3%, respectively. On the brain tumor dataset, mIoU, Dice, VOE, RVD and Recall are increased by 3.69%,2.93%, 6.23%, 4.37% and 4.06%,respectively, which proves the effectiveness of the method.
Zhou et al. (Mon,) studied this question.