Glance Once-Touch Once (GOTO): Physics-informed visuo-tactile human-like grasping for embodied robotic manufacturing

Adaptive robotic grasping in unstructured manufacturing remains challenging due to the need to handle objects with varying geometries, material stiffness, and functional constraints. However, existing methodologies typically decouple posture detection from stability control. Specifically, geometric-centric algorithms often suffer from semantic blindness, prioritizing geometric fit over functional intent. Concurrently, stability strategies predominantly rely on binary classification rather than adaptive physical awareness, lacking the explicit physical commonsense required to manipulate diverse materials. This isolation of the semantic “where” from the physical “how” leads to severe safety risks and efficiency bottlenecks. To bridge these gaps, this paper presents a physics-informed visuo-tactile adaptive grasping framework that emulates the human “Glance Once-Touch Once” mechanism, seamlessly integrating semantic perception with adaptive force control. Specifically, a task-oriented fine-tuned affordance network is designed to precisely localize functional grasping areas regardless of object shape. Subsequently, a physics-informed asymmetric tri-stream network is designed to integrate visual context, tactile feedback, and LLM-derived physics priors to regress the optimal clamping force. Experimental results demonstrate state-of-the-art performance with a grasp categorization mIoU of 86.08% and a force control MAE of 15.48 units. Extensive real-world validation, involving 200 trials across 20 diverse object categories, yielded an 89% success rate. Furthermore, a collaborative gear reducer assembly case study confirms that by endowing robots with comprehensive human-like awareness, our framework significantly enhances efficiency and safety, offering valuable insights for the large-scale application of embodied robots in unstructured real-world manufacturing environments.