Efficient Stacking and Grasping in Unstructured Environments

Abstract Robotics has been booming in recent years. Especially with the development of artificial intelligence, more and more researchers have devoted themselves to the field of robotics, but there are still many shortcomings in the multi-task operation of robots. Reinforcement learning has achieved good performance in manipulator manipulation, especially in grasping, but grasping is only the first step for the robot to perform actions, and it often ignores the stacking, assembly, placement, and other tasks to be carried out later. Such long-horizon tasks still face the problems of expensive time, dead-end exploration, and process reversal. Hierarchical reinforcement learning has some advantages in solving the above problems, but not all tasks can be learned hierarchically. This paper mainly solves the complex manipulation task of continuous multi-action of the manipulator by improving the method of hierarchical reinforcement learning, aiming to solve the task of long sequences such as stacking and alignment by proposing a framework. Our framework completes simulation experiments on various tasks and improves the success rate from 78.3% to 94.8% when cleaning cluttered toys. In the stacking toy experiment, the training speed is nearly three times faster than the baseline method. And our method can be generalized to other long-horizon tasks. Experiments show that the more complex the task, the greater the advantage of our framework.