Intra- and inter-rater reliability in log volume estimation based on LiDAR data and shape reconstruction algorithms: a case study on poplar logs

Producing reliable log volume data is an essential feature in an effective wood supply chain, and LiDAR sensing, supported by portable platforms, is a promising technology for volume measurements. Computer-based algorithms like Poisson interpolation and Random Sampling and Consensus (RANSAC) are commonly used to extract volume data from LiDAR point clouds, and comparative studies have tested these algorithms for accuracy. To extract volume data, point clouds require several post-processing steps, while their outcome may depend largely on human input and operator decision. Despite the increasingly number of studies on accuracy limits, no paper has addressed the reliability of these procedures. This raises at least two questions: (i) Would the same person, working with the same data and using the same procedures get the same results? And (ii) How much would the results deviate when different people process the same data using the same procedures? A set of 432 poplar logs placed on the ground and spaced about 1 m apart, was scanned by a professional mobile LiDAR scanner in groups; the first 418 logs were then individually scanned using an iPhone-compatible app, with the remainder being excluded from this part of the study due to field time constraints and all the logs were manually measured to get the reference biometric data. Three researchers with different experiences processed the datasets produced by scanning twice, following a protocol that included shape reconstruction and volume calculation using Poisson interpolation and RANSAC algorithm for cylinders and cones. The intra- and inter-rater reliability were evaluated using a comprehensive array of statistical metrics. The results show that the most reliable estimates correlate with a greater experience. The Cronbach’s alpha metric at the subject level was high, with values of 0.902–0.965 for the most experienced subject, and generally indicated moderate to excellent intra-rater reliabilities. Moreover, working with Poisson interpolation and RANSAC cylinder shape reconstruction, respectively, indicated a moderate to excellent reliability. For the Poisson interpolation algorithm, the Intraclass Correlation Coefficient (ICC) ranged from 0.770 to 0.980 for multi-log datasets, and from 0.924 to 0.972 for single log datasets. For the same type of input datasets, the ICC varied between 0.761 and 0.855 and from 0.839 to 0.908 for the RANSAC cylinder, and from 0.784 to 0.869 and 0.843 to 0.893 for the RANSAC cone shape reconstruction algorithms, respectively. These values indicate a moderate to excellent inter-rater reliability. Similar to Cronbach’s alpha, the Root Mean Square Error (RMSE) was related in magnitude to the ICC. The results of this study indicate that, for improved reliability and efficiency, it is essential to automate point cloud segmentation using advanced machine learning and computer vision algorithms. This approach would eliminate the subjectivity in segmentation decisions and significantly reduce the time required for the process.