Answers are grounded in this paper and cite their sources.
Analysis showcases improved worker training with AR for component identification, suggesting efficiency in smart manufacturing settings.
Current smart manufacturing computational tools and optimization methods leverage advances from artificial intelligence (AI), augmented / mixed/virtual reality, and predictive modeling to meet factory production goals. With significant developments in computer graphics tools, deep learning methods, and immersive experiences (metaverse), there is a need to study its seamless adaptation to AR-based manufacturing-specific scenarios, especially for worker training. In this work, we develop an augmented reality-based worker training framework aimed at helping novice technicians with complex machine component identification in several smart factory tasks and shop-floor training environments. A Deep-Learning (DL) semantic segmentation network U-Net, prominently applied in several manufacturing use cases, is enhanced for detailed pixel-level component identification. We formulated an automatic synthetic image generation algorithm that uses computer-aided design (CAD) models generated using Siemens NX to create images and automatic labels for training the neural network. These images are custom-rendered with variable lighting and randomized orientations to mimic real-life occlusions and improve data diversity for neural network training. In this approach, arbitrary textured materials are assigned to the 3D objects through Blender, an open-source computer graphics software using Python programming in the input scene environment. The background for these images is set from the multiple real-world environment scenes. A custom Python script integrated into Blender captures the image snapshots through the built-in camera system. It automatically separates render and mask images, thus saving time on the tedious manual labeling process. We pass the virtual images (both render and mask images) to the DL pipeline for training. After training the data through Pytorch and CUDA parallel computing, the Open Neural Network Exchange (ONNX) format output model is imported to the AR app. The application is developed using the Unity 3D game engine platform and deployed on Android. Specifically, the Unity Barracuda toolkit maps the input and output tensors as rendering textures and recognizes components in colored pixels. We present a prototype three-a stage automotive engine assembly case study to evaluate the proposed framework’s effectiveness. The engine components include tools (mallet, chisel, wrench), machine tooling equipment (nuts, bolts, Allen keys, screwdriver), and assembly equipment (engine piston, cylinder head, and camshaft). The 3D-printed components mimicking engine parts are identified in real-time along their exact pixel-level contours in the AR app. This framework can extend AR-based methods to visualize components in complex wiring, cluttered environments, and intricate assemblies on the factory floor, reducing the training time performed through conventional approaches and providing an interactive training approach through gamification.
No takes yet. Share an insight, caveat, or question.
Deshpande et al. (2024) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: