SPADe: Spatial Plaid Attention Decoder for Semantic Segmentation of Street Views

The decoder is a key component in deep networks for the semantic segmentation of street views. The existing methods rely on the limited receptive field for feature extraction without considering the contextual information, which could lead to errors in understanding complex scenes. Moreover, a balance of contextual information and computational cost must be considered to meet the needs of real-world applications. To address these problems, we introduce a Spatial Plaid Attention Decoder network, which uses a lightweight decoder with Spatial Plaid Attention to perform highly efficient operations for semantic segmentation. With approximately 4 million parameters (9.75% of the UPerNet), our decoder achieves state-of-the-art performance on public datasets such as Cityscapes and ADE20K, with 84.84% and 54.0% mIoU, respectively. In addition, our method reduces the total Flops by 34.95% and 32.85%, respectively. We demonstrate how contextual information helps the network in object recognition and how object features and contextual features contribute to the scene segmentation and recognition.