Vision-based human–robot object handover in dynamic environments: a safety-aware framework with real-time anthropomorphic planning

Purpose Dynamic human–robot object handover faces challenges such as insufficient generalization capability, limited real-time performance and safety concerns in practical applications. This paper aims to propose a vision-based human–robot object handover system to execute object handovers from a human worker to a robot more safely, efficiently and fluently in dynamic scenarios. Design/methodology/approach The proposed system takes images captured by a single RGB-D camera to complete object handover scene understanding. Putting a high emphasis on efficiency and safety, a novel lightweight EMA-UNet network is designed for efficient and high-precision hand segmentation and GR-ConvNet network is used for grasp detection. To maximize fluency of human–robot object handover, an anthropomorphic trajectory planning method based on dynamic movement primitives (DMPs) is used in this system. Experiments were conducted on the AUBO E5 robotic arm platform, with system performance validated through 200 handover tasks performed by five participants. Findings The experimental results show that the system achieves an average handover success rate of 84% across diverse objects, with a 1% probability of collision. Compared with U-Net network, the proposed EMA-UNet network reduces the number of parameters by 98.5% while maintaining an IoU of 98.04%. The system’s inference time is 0.16 s, which can meet real-time requirements in dynamic human–robot object handover. Originality/value In this work, a vision-based human–robot object handover system is proposed by combining lightweight hand segmentation, multimodal grasp detection and DMP-based trajectory planning. The results demonstrate that the proposed system can facilitate a dynamic, efficient, safe and fluent human–robot object handover.