Recent advances in multimodal technologies and tactile sensing have spurred increasing interest in visuo-tactile fusion for enhancing in-contact robotic grasping performance. However, existing visual-tactile datasets and methods often lack explicit 3D spatial alignment between object visual shape and robot tactile contact. This absence of explicitly aligned 3D geometric representation limits effective multimodal reasoning in contact-rich grasping scenarios. To address this limitation, we present 3DA-VTG, a large-scale dataset designed to facilitate explicitly aligned 3D visuo-tactile learning. The dataset contains 440 K annotated grasp datapoints with an average density of 5 K grasps per object. All datapoints are collected under simulated object handover scenarios that enable diverse poses and consistent 3D alignment. Leveraging this dataset, we further propose a unified 3D visuo-tactile perception framework for grasp stability prediction. Visual and tactile observations are first represented in a shared 3D space, followed by shape completion to improve geometric completeness. On top of this representation, we develop SGA-GSN, a dual-branch network that integrates geometry-aware cross-attention with pyramid downsampling to fuse shape and contact information. Experimental results on 3DA-VTG demonstrate that the proposed method achieves 81.9% and 79.8% grasp stability prediction accuracy on seen and unseen objects, respectively, yielding an average improvement of around 4.0% over recent 2D non-aligned baselines. These results highlight the importance of explicit 3D alignment for grasp stability prediction. Furthermore, real-world robotic experiments on rigid objects achieve 74.0% prediction accuracy, indicating that explicit 3D visuo-tactile alignment is practically deployable for grasp stability assessment. Our datasets, APIs, and source code are publicly available at https://github.com/Yiju1213/3DA-VTG .
Tao et al. (Sun,) studied this question.