Bridging cognition, reasoning and action for polymorphic embodied agents towards human-heterogeneous multi-robot collaborative assembly with vision-language model and digital twin

With the advent of Industry 5.0, human-robot collaborative assembly has become a core production paradigm that combines human cognitive flexibility with robotic operational efficiency. Single-robot systems cannot meet the diverse requirements of complex assembly tasks. Therefore, collaboration between human operators and heterogeneous multi-robot teams is required for flexible and efficient manufacturing. However, current collaborative systems are limited by fragmented perception-reasoning-execution pipelines. Poor generalization of general-purpose models in industrial environments is also observed. Additionally, integrated safety mechanisms for dynamic shared workspaces remain insufficient. These challenges indicate the need for embodied intelligence, through which agents can perceive, reason, and act seamlessly in physical environments. To address these issues, a vision-language model (VLM)-enabled, digital twin-assisted polymorphic embodied agent framework was proposed in this paper. A complete closed loop of embodied cognition, embodied reasoning, and embodied execution was established for human-heterogeneous multi-robot collaborative assembly. In the embodied cognition layer, a domain-specific VLM was developed to achieve robust real-time perception of human behaviors, assembly components, and environmental states, with integrated depth information for accurate spatial localization and collision detection. In the embodied reasoning layer, a digital twin system with bidirectional virtual-to-real mapping enables virtual pre-simulation and optimal task scheduling that effectively leverages the complementary strengths of human and robots. In the embodied execution layer, the model directly generates executable control code for heterogeneous robots, which is pre-verified in the digital twin environment before physical deployment. Comprehensive experimental results demonstrate that the proposed framework outperforms traditional visual algorithms and mainstream general-purpose vision-language models on all evaluation benchmarks. The system achieves three tiered latency indicators for decoupled functional modules. It delivers ultra-low single-frame perception delay below 150 ms for environmental safety sensing, and attains outstanding reasoning accuracy paired with an online task reasoning latency of 0.75 s to support real-time human-robot closed-loop interaction. The 4.8 s full assembly code generation delay only takes effect during offline pre-generation before task startup. This offline process is isolated from the online control loop and imposes no negative impact on the system’s real-time dynamic response and obstacle avoidance capacity. Continuous robotic operation was ensured. Human comfort and safety were maintained through differentiated task allocation. This paper provides a practical and effective solution for intelligent and safe collaborative assembly. It also offers important implications for the development of next-generation intelligent manufacturing systems.