Influence of temperate forest autumn leaf phenology on segmentation of tree species from UAV imagery using deep learning

Remote sensing of forests has become increasingly accessible with the use of unoccupied aerial vehicles (UAV), along with deep learning, allowing for repeated high-resolution imagery and the capturing of phenological changes at larger spatial and temporal scales. In temperate forests during autumn, leaf senescence occurs when leaves change colour and drop. However, the influence of leaf senescence in temperate forests on tree species segmentation using a Convolutional Neural Network (CNN) has not yet been evaluated. Here, we acquired high-resolution UAV imagery over a temperate forest in Quebec, Canada on seven occasions between May and October 2021. We segmented and labelled 23,000 tree crowns from 14 different classes to train and validate a CNN for each imagery acquisition. The CNN-based segmentation showed the highest F1-score (0.72) at the start of leaf colouring in early September and the lowest F1-score (0.61) at peak fall colouring in early October. The timing of the events occurring during senescence, such as leaf colouring and leaf fall, varied substantially between and within species and according to environmental conditions, leading to higher variability in the remotely sensed signal. Deciduous and evergreen tree species that presented distinctive and less temporally-variable traits between individuals were better classified. While tree segmentation in a heterogenous forest remains challenging, UAV imagery and deep learning show high potential in mapping tree species. Our results from a temperate forest with strong leaf colour changes during autumn senescence show that the best performance for tree species segmentation occurs at the onset of this colour change. • Effect of leaf phenology on tree segmentation from drone imagery is not well known. • U-Net semantic segmentation yieled good tree-cover segmentation for most species. • The best performance was found at the onset of autumn colours. • Species with less heterogenous crowns between individuals were better segmented. • A dataset of 23,000 tree crown annotations over a growing season was generated.