Improvement of Panoptic Segmentation Model Based on Path Aggregation Feature Pyramid Network and Attention Mechanism

ABSTRACT As a core task in image understanding, panoptic segmentation integrates instance‐level segmentation for countable objects, referred to as things, and semantic‐level segmentation for uncountable regions, referred to as stuff, thereby enabling comprehensive pixel‐level parsing of image scenes. The original Panoptic FPN relies on unidirectional top–down feature propagation: its high‐resolution shallow features lack sufficient semantic information, and no adaptive filtering is applied to backbone features. These limitations restrict the segmentation accuracy for small objects and those with overlapping features. To address this issue, this study proposes improvements to the Panoptic FPN panoptic segmentation model, with the goal of enhancing segmentation performance and accuracy. Specifically, the traditional feature pyramid network in the original model is replaced with a path aggregation feature pyramid network, hereafter referred to as PAFPN, which transforms the information flow from unidirectional to bidirectional propagation. Additionally, a channel‐spatial attention mechanism is incorporated after PAFPN completes feature fusion on the aggregated pyramid features, so as to obtain higher‐quality features. Experimental results on the COCO2017 dataset show that the improved Panoptic FPN model achieves a 0.556% increase in Panoptic quality, denoted as PQ, compared with the original model—this verifies the effectiveness of the proposed improvements. Compared with mainstream panoptic segmentation models, the improved model also achieves favorable performance.