Answers are grounded in this paper and cite their sources.
Effective operational training for production equipment is crucial to ensure personnel safety and realize the human-centric vision of Industry 5.0. However, existing Mixed Reality (MR)-based training methods in manufacturing workshops face two major challenges: difficulty in the semantic perception of equipment and lack of context-aware knowledge responses. To address these challenges, this paper proposes an interactive virtual-physical safety training for manufacturing workshops via vision and Large Language Models (LLMs). First, a methodological framework is established. Next, a machine vision-based semantic segmentation method combined with Vuforia multi-target recognition is employed to enable semantic understanding and 3D virtual-physical mapping of training equipment. Subsequently, a knowledge-grounded question answering method is introduced by integrating knowledge graphs and LLMs to support context-aware training feedback. Finally, a prototype safety training system is developed on HoloLens 2 and experimentally validated. Results show that, compared with traditional paper-based training, the proposed method shows potential to improve trainees’ performance in short-term tasks, particularly in skill-based training tasks under the present experimental conditions. These findings demonstrate its potential as a technical enabler for human-centered industrial safety training for workshop operators.
No takes yet. Share an insight, caveat, or question.
Wang et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: