Development of an Image-Based Framework for Quantitative Extraction of Welding Process Features in Arc Welding Using Semantic Segmentation

Objective feedback is important for improving welding training and reducing dependence on experienced instructors. This study proposes an image-based sensing framework for extracting motion and geometric features from videos acquired using a simple visualization system mounted inside a welding helmet during semi-automatic gas metal arc welding (GMAW) and manual gas tungsten arc welding (GTAW). The welding videos were acquired in real time, whereas the subsequent image-processing and quantitative feature-extraction procedures were performed offline. This visualization system consists of an instructor-side unit and a welder-side unit equipped with a prototype compact camera. Welding images were trimmed to 288 × 288 pixels and classified into eight regions for GMAW and six regions for GTAW using a U-Net-based semantic segmentation model. The segmented regions in GMAW included the arc, molten pool, groove, torch, wire, overlap regions, and background, whereas those in GTAW included the arc, bead, groove, electrode, filler wire, and background. Groove edges were detected using Canny edge detection and the Hough transform. The wire-tip position in GMAW and the electrode-tip position in GTAW were estimated using the arc centroid as an image-based positional proxy and normalized by the detected groove width to obtain normalized wire-tip and electrode-tip motion, respectively. In addition, the molten-pool width in GMAW and the bead width in GTAW were extracted and normalized using the same procedure to obtain groove-normalized geometric features. For the GMAW dataset, the segmentation model achieved a validation accuracy of 95.2% and a validation mean intersection over union (mIoU) of approximately 67.1%. For the GTAW dataset, the corresponding validation accuracy and validation mIoU were 96.6% and approximately 85.9%, respectively. The results showed that the proposed framework can successfully extract normalized wire-tip motion and molten-pool width in GMAW, as well as normalized electrode-tip motion and bead width in GTAW, from welding videos acquired from different welders. These findings demonstrate the potential of the proposed framework to extract quantitative motion and geometric features from welding videos, providing useful data for future welding training support and welding behavior analysis.