Automated 3D Radiological and Microscopy image analysis enables quantitative therapeutics assessment. However, accurate segmentation and smaller footprints are needed. To address these issues, the current study introduces the Multi-Aperture Transformer-Convolutional Network (MAFTC-Net) with a custom loss function incorporating principal curvatures to penalize topological dissimilarities. The multi-aperture modality represents each image patch as a series of images with smaller apertures along the same central field of view. Each aperture has a pyramid representation, with the advantage of maintaining the original image resolution. The MAFTC-Net encoder comprises four Swin Transformers, where a convolutional network processes the output of each transformer to accentuate local features within a global context. Next, the outputs of the transformer and convolution blocks are integrated with a 3D fusion block. MAFTC-Net was trained on the Synapse multi-organ, ACDC, and 3D Organoid datasets, achieving an average Dice score of 90.18±0.04, 93.55±0.05, and 95.13±0.03, respectively. The improved performance also comes with the added benefits of reduced complexity with 38.2 million parameters.
Mohammed et al. (Mon,) studied this question.