Collapse and collision aware grasping for cluttered shelf picking

In modern smart factories, automated shelf picking must deliver high throughput, flexibility, and safe human–robot collaboration. In these environments, efficient and safe retrieval of stacked objects is a significant challenge due to complex spatial dependencies and structural inter-dependencies. Traditional vision-based methods excel at object localization but often lack the physical reasoning required to predict the consequences of extraction, leading to unintended collisions and collapses. This paper proposes a collapse and collision-aware grasp planner that integrates dynamic physics simulations for robotic decision-making. Using a single image and depth map, an approximate 3D representation of the scene is reconstructed in a simulation environment, enabling the robot to evaluate different retrieval strategies before execution. Two approaches: 1) heuristic-based and 2) physics-based are proposed for both single-box extraction and shelf clearance tasks. Extensive real-world experiments on structured and unstructured box stacks, along with validation using datasets from existing databases, show that our physics-aware method significantly improves efficiency and success rates compared to baseline heuristics. A video demonstrating the real-world implementation of our proposed system is available at: https://youtu.be/GBWMiNIHUlU .