Generation of grasping movements for multiple objects using deep predictive learning

In robot motion generation using machine learning, the presence of multiple similar objects in camera images can cause ambiguity in selecting the appropriate target, potentially leading to incorrect or failed grasping motions. To address this issue, this study proposes a method combining object detection using YOLO with deep predictive learning. By masking unnecessary objects detected by YOLO, the network’s attention is guided toward the proper target. The processed images, along with the robot’s joint angles and classification results related to the object’s orientation, are input into a Spatial Attention Recurrent Neural Network (SARNN) to generate the desired motion. The effectiveness of the proposed method was verified using OpenMANIPULATOR-X. We confirmed that even in environments with multiple similar objects, the grasp success rate improves with a small amount of training data.