Digital Twin-enabled adaptive robotics: multi-agent reasoning over language, vision, and structured database

The growing demand for customized products challenges traditional automation systems that rely on repetitive and predefined operations. This paper presents an adaptive robotic framework combining multi-agent reasoning, camera vision, and natural language interaction for flexible automation in unstructured environments. The framework builds on a digital twin architecture supported by a structured database managing task parameters, operational data, and inter-agent communication. Its modular design enables adaptation to new tasks by activating specific agents based on user instructions and environmental conditions, without manual reprogramming. Two application cases demonstrate the framework’s adaptability: hospital laboratory slide-sorting and block-stacking. Both use vision-based detection and automatic generation of virtual environments mirroring real-world scenes in simulation. Randomized trials evaluate architectural feasibility and system behavior, achieving an overall success rate of 75%. The results demonstrate a robust and extensible approach to adaptive automation, advancing Industry 5.0 through human-centric, intelligent, and context-aware robotic systems for dynamic environments.