Human–machine interaction and implementation on the upper extremities of a humanoid robot

Abstract Estimation and tracking the various joints of the human body in a dynamic environment plays a crucial role and it is a challenging task. Based on human–machine interaction, in the current research work the authors attempted to explore the real-time positioning of a humanoid arm using a human pose estimation framework. Kinect depth sensor and media pipe framework are used to obtain the three-dimensional position information of human skeleton joints. Further, the obtained joint coordinates are used to calculate the joint angles using the inverse kinematics approach. These joint angles are helpful in controlling the movement of the neck, shoulder, and elbow of a humanoid robot by using Python-Arduino serial communication. Finally, a comparison study was conducted between the Kinect, MediaPipe, and real-time robots while obtaining the joint angles. It has been found that the obtained result from the MediaPipe framework yields a minimum standard error compared to Kinect-based joint angles.