Dynamic Attention Analysis of Body Parts in Transformer-Based Human–Robot Imitation Learning with the Embodiment Gap

In imitation learning between humans and robots, the embodiment gap is a key challenge. By focusing on a specific body part and compensating for the rest according to the robot’s size, the embodiment gap can be overcome. In this paper, we analyze dynamic attention to body parts in imitation learning between humans and robots based on a Transformer model. To adapt human imitation movements to a robot, we solved forward and inverse kinematics using the Levenberg–Marquardt method and performed feature extraction using the k-means method to make the data suitable for Transformer input. The imitation learning process is carried out using the Transformer. UMAP is employed to visualize the attention layer within the Transformer. As a result, this system enabled imitation of movements while focusing on multiple body parts between humans and robots with an embodiment gap, revealing the transitions of body parts receiving attention and their relationships in the robot’s acquired imitation movements.