Automatic tree-level based forest inventories retrieval via ultra-high resolution UAV images and deep learning

Forest inventories play an essential role in managing and protecting forest resources as well as quantifying carbon stocks. Recent advances in Uncrewed Aerial Vehicles (UAVs) have enhanced capabilities for efficiently monitoring forest dynamics across large geographic areas. RGB cameras are typically preferred for rapid and scalable forest inventory missions owing to three distinct advantages, including low cost, ease of use, and high resolution. However, compared with multispectral or hyperspectral sensors, the limited spectral signals of RGB cameras pose challenges for tree crown detection and classification. The ability of deep learning methods to capture structural and contextual cues from imagery helps alleviate some of the limitations of RGB data. In this study, we propose an Individual Tree Crown (ITC)-based framework leveraging UAV data and advanced deep learning models for inventories of individual trees in dense and natural forests. First, we develop the ITC-based Multi-Task Convolutional Neural Network (ITCMNet), which incorporates multi-scale contexts to simultaneously and accurately identify individual tree crowns, discriminate tree species, and assess tree vitality. Second, structural parameters for each individual crown are extracted to estimate forest carbon storage using species-specific allometric models. Unlike conventional pixel-based methods, our proposed ITCMNet enables precise forest investigations at the ITC level, enhancing both performance and interpretability. We collected a comprehensive dataset consisting of 2456 ultra-high resolution (1.6 cm) UAV RGB images and 27,160 labeled trees across 105 plots distributed in three dense forests and one city park in Germany to evaluate our framework. The ITCMNet demonstrated robust tree crown delineation performance, achieving an F1 score of 0.81. Additionally, our method attained an F1 score in species classification (i.e., 0.54 for angiosperms and 0.76 for gymnosperms) and vitality identification (0.66). Utilizing precise tree parameters, species information, and species-specific allometric models, our carbon storage estimation surpassed current satellite-based carbon products. The carbon stock estimation achieved an R 2 of 0.83 and the carbon storage range in the Bamberg forests is approximately 50 to 110 Mg C/ha. These results show that our proposed framework provides detailed, cost-effective forest inventories, highlighting its potential to support various downstream forestry applications. The dataset and source code are available ( https://www.dlr.de/en/eoc/about-us/remote-sensing-technology-institute/photogrammetry-and-image-analysis/public-datasets/bamforests ; https://github.com/WendyFan52/ITCMNet ).