Pallet Detection and Localization Based on RGB Image and Point Cloud Data for Automated Forklift

Pallet detection and localization is a fundamental task in the process of logistics automation. Accurate estimation of the pallet's pose contributes to enhanced efficiency and safety. This paper presents a pallet detection and localization method based on the object detection model YOLOv5s and point cloud processing. The proposed method comprises three stages: 1) point cloud cropping based on point cloud quantity distribution in depth direction and the pallet bounding box output by object detection model on the RGB image to reduce computational data volume; 2) pallet front surface fitting using the dynamic parameter iteration MSAC algorithm to solve the pallet's deflection angle; and 3) pallet entry holes localization and pallet center point calculation based on image processing. The proposed method was tested using RGB-images and point cloud data of the pallet on a shelf positioned in front of the forklift, where the data was captured by a structured-light 3D camera. The poses of the pallets are set into two groups with different deflection angles and different distances. The results indicate that the computational error of the method in terms of angle is within 0.4 degrees, and the error in distance is controlled within 1 centimeter, which has a certain degree of accuracy and stability.