Answers are grounded in this paper and cite their sources.
Shows improved task efficiency in object manipulation using a mixed reality framework, suggesting enhanced remote robot operation.
This work presents a Mixed Reality (MR) teleoperation framework that enables intuitive control of a bimanual robot. The system integrates immersive MR with differential kinematics to map the operator’s movements to the robot in real time, while providing multiple visual feedback modalities (2D video stream, colored point cloud rendering, and stereoscopic stream) for enhanced spatial perception. A key feature of the proposed framework is the Point-and-Grasp mode, which implements a complete grasping pipeline grounded in geometric analysis and scoring criteria. In this mode, the user selects a target object via a ray-cast in the MR interface; the system then segments the object, generates candidate grasps, evaluates them according to geometric suitability, and autonomously executes the most promising one. A user study was conducted to assess usability, task efficiency, and subjective workload across different visual feedback modalities in object-manipulation tasks. To evaluate grasping performance, a custom dataset of forty cluttered tabletop scenes was created to ensure consistent experimental conditions. The proposed grasp selection strategy achieved the highest success rate among the evaluated methods under the target-conditioned protocol, showing consistent behavior in cluttered tabletop scenes. Overall, the results indicate that immersive teleoperation combined with selective grasp autonomy can improve efficiency, usability, and accessibility in remote robot manipulation. To support reproducibility and further research, the source code associated with the grasping pipeline is publicly available at: https://github.com/MiguelGarciaUVa/PointAndGrasp .
No takes yet. Share an insight, caveat, or question.
García-Gómez et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: