3D computer vision and machine learning based technique for high throughput cotton boll mapping under field conditions

Abstract. This study presented a multi-view imaging system with consumer-graded digital cameras to acquire images and reconstruct 3D model based on structure from motion principle in the field. A 3D point cloud data processing pipeline was proposed following three steps. First, the ground plane was removed using the RANSAC algorithm; then, a support vector machine based model was trained using color features for cotton boll segmentation from plants. At last, a 3D DBSCAN based method was developed to detect individual bolls from the segmented boll voxels. Experiments with cotton plots showed that good quality 3D model can be reconstructed and the proposed 3D boll mapping methods achieved an accuracy of around 90%, and the squared Pearson correlation was 0.95 between the sensor measurement and the ground truth. The system not only successfully estimated the total number of cotton bolls, but also provided location information for each individual bolls. This was a significant contribution compared to 2D image based methods. The 3D boll mapping information was useful for monitoring crop growth and yield prediction.