Evaluating Multispectral Data Fusion for Dense Instance Segmentation in Vegetation and Artificial Objects Point Clouds

Abstract. Multispectral data improves instance segmentation in digital agriculture by combining geometric and spectral information to distinguish complex natural features. While geometric information captures structural details, it often falls short when dealing with complex natural features that exhibit high spectral similarity, rather than due to limitations inherent to geometric representation itself. This work presents a feasibility analysis of instance segmentation using a spectral point cloud. A combination of spectral bands is selected based on class separability and proximity to a normal distribution as estimated by the Shapiro–Wilk test. The aim is to identify the minimum number of bands required to produce optimum results. For the normality analysis, Euclidean magnitude normalisation was applied, and it was also used alongside standard scaling to support the Multilayer Perceptron (MLP) for classification and segmentation. To refine the MLP predictions and consolidate instance labels, a graph-based post-processing step was applied, linking each point to its nearest neighbours and using a majority-voting scheme, resulting in spatially coherent clusters and refining the MLP predictions. The results demonstrate that multispectral data can reliably segment individual objects, with ten spectral bands being sufficient to achieve highly satisfactory segmentation and accurately delineate natural features such as leaves and tree trunks. Further increasing the number of bands improved spectral definition even more, with 14 bands achieving the highest performance across all metrics (mIoU: 96.59%; AP50: 96.14%). These findings highlight the strong potential of multispectral point clouds for precise and scalable object-level segmentation in agricultural environments.