A Semantic‐to‐Motion Digital Twin Framework for Expressive Industrial Avatars in Telepresence

Remote industrial assistance is increasingly mediated through robotic avatars, yet existing telepresence systems provide limited support for non‐verbal behaviors such as hesitation, urgency, or emphasis. Manually controlling such expressive motion can increase operator workload. We present a feasibility and system‐integration study of a semantic‐to‐motion telepresence pipeline that interprets expert utterances and scene context into parameterized expressive robot behaviors. The system combines visual grounding with language‐based intent inference to produce structured motion descriptors, which are mapped to procedural motion primitives executed in a unity‐based digital twin and streamed to a physical robot via ROS. As the semantic control vocabulary for this prototype, we define SAEH (show, alert, encourage, hesitate), a four‐class operational vocabulary derived from a thematic review of 104 HRI papers. We implement the approach on a Niryo Ned 1 manipulator and show that it can generate and execute kinematically distinct expressive motion profiles, including deictic, warning, supportive, and hesitant behaviors. The proposed framework partially separates communicative intent from robot‐specific execution through parameterized procedural primitives, providing a technical foundation for context‐aware expressive telepresence; however, a measured mean end‐to‐end latency of 10 s currently limits its use to supervisory and asynchronous task loops rather than continuous live interaction.