A convolutional neural network architecture designed for the automated survey of seabird colonies

Abstract Satellite imagery is now well established as a method of finding and estimating the abundance of Antarctic penguin colonies. However, the delineation and classification of penguin colonies in sub‐meter satellite imagery has required the use of expert observers and is highly labor intensive, precluding regular censuses at the pan‐Antarctic scale. Here we present the first automated pipeline for the segmentation and classification of seabird colonies in high‐resolution satellite imagery. Our method leverages site‐fidelity by using images from previous years to improve classification performance but is robust to georegistration artifacts imposed by misalignment between sensors or terrain correction. We use a segmentation network with an additional branch that extracts the useful information from the prior mask of the input image. This prior branch provides the main model information on the location and size of guano in a prior annotation yet automatically learns to compensate for potential misalignment between the prior mask and the input image being classified. Our approach outperforms the previous approach by 44%, improving the average Intersection‐over‐Union segmentation score from 0.34 to 0.50. While penguin guano remains a challenging target for segmentation due to its indistinct and highly variable appearance, the inclusion of prior information represents a key step toward automated image annotation for population monitoring. Moreover, this method can be adapted for other ecological applications where the dynamics of landscape change are slow relative to the repeat frequency of available imagery and prior information may be available to aid with image annotation.