Human Motion Capture Based on Kinect and IMUs and Its Application to Human-Robot Collaboration

In this paper, a nonlinear optimization solver is proposed to establish the human upper body motion capture system, including the orientation of each bone joint and the global position of the hip bone. 5 IMUs and one Kinect are used, the rotation data and acceleration data of IMUs, Kinect skeleton position data and human pose prior are fused in a non-linear optimization way. The method proposed in this paper combines the characteristics of Kinect data with high accuracy and IMU data with stability, which can make up for the accuracy of human pose capture in dynamic, magnetically disturbed and occlusion conditions. In order to verify the practicability of the human motion capture system proposed in this paper, the system is firstly applied to real-time human motion capture, then a simple human-robot collaboration (HRC) case is implemented, which is very common in modern collaborative medical field. The experimental results show that the proposed human motion capture system can achieve stable human motion measurement, which can meet the needs of HRC scenarios.