Intra-and Inter-rater Reliability in Log Volume Estimation Based on LiDAR Data and Shape Reconstruction Algorithms: A Case Study on Poplar Logs

Stelian Alexandru Borz^1,2*Gabriel Osei Forkuo¹

• ¹Department of Forest Engineering, Forest Management Planning and Terrestrial Measurements, Faculty of Silviculture and Forest Engineering, Transilvania University of Brasov, Brasov, Romania
• ²Transilvania University of Brașov, Brasov, Brasov, Romania

Producing reliable log volume data is an essential feature in an effective wood supply chain, and LiDAR sensing is a promising technology for volume measurements. Despite the increasingly number of studies on accuracy limits, no paper has addressed the reliability of estimates. This raises at least two questions: (i) Would the same person, working with the same data and by using the same procedures get the same results? and (ii) How much would the results deviate when different people process the same data using the same procedures? A set of 432 poplar logs placed on the ground and spaced about 1 meter apart, was scanned by a professional mobile LiDAR scanner as groups; the first 418 logs were then individually scanned using an iPhone-compatible app, and all the logs were manually measured to get the reference biometric data. Three researchers processed the datasets produced by scanning twice, following a protocol that included shape reconstruction and volume calculation using Poisson interpolation and RANSAC algorithm for cylinders and cones. Then, intra-and inter-rater reliability were evaluated. The results show that reliability of estimates correlates with experience. The Cronbach's alpha metric at the subject level was high, with values of 0.902 to 0.965 for the most experienced subject, and generally indicated moderate to excellent reliabilities. Working with Poisson interpolation and RANSAC cylinder shape reconstruction, indicated a moderate to excellent reliability. For the Poisson interpolation algorithm, the Intraclass Correlation Coefficient (ICC) ranged from 0.770 to 0.980 for multi-log datasets, and from 0.924 to 0.972 for single log datasets. For the same type of datasets, the ICC varied between 0.761 to 0.855 and from 0.839 to 0.908 for the RANSAC cylinder, and from 0.784 to 0.869 and 0.843 to 0.893 for the RANSAC cone shape reconstruction algorithms. These values indicate a moderate to excellent inter-rater reliability. Similar to Cronbach's alpha, the Root Mean Square Error (RMSE) was related in magnitude to the ICC. The results of this study indicate that, for improved reliability and efficiency, it is essential to automate point cloud segmentation using advanced machine learning and computer vision algorithms.