Semantic Segmentation of Aerial Images in Urban Areas With Class-Specific Higher-Order Cliques

Abstract. In this paper we propose an approach to multi-class semantic segmentation of urban areas in high-resolution aerial images with classspecific object priors for buildings and roads. What makes model design challenging are highly heterogeneous object appearances and shapes that call for priors beyond standard smoothness or co-occurrence assumptions. The data term of our energy function consists of a pixel-wise classifier that learns local co-occurrence patterns in urban environments. To specifically model the structure of roads and buildings, we add high-level shape representations for both classes by sampling large sets of putative object candidates. Buildings are represented by sets of compact polygons, while roads are modeled as a collection of long, narrow segments. To obtain the final pixel-wise labeling, we use a CRF with higher-order potentials that balances the data term with the object candidates. We achieve overall labeling accuracies of > 80%.