AI-enhanced vision system for dispensing process monitoring and quality control in manufacturing of large parts

Assembly is one of the cornerstones of today’s manufacturing; thus, efficient automated assembly systems have become a necessity in the recent past. Challenges that have traditionally been present in such systems involve positioning accuracy, stable gripping, accommodation of several variants by the same resources and prevention of error aggregation. In robot-based applications the detection of parts and their components is a common task, but when it comes to large objects which cannot be portrayed in single camera frames, the complexity becomes significantly larger. This paper presents an AI based perception system which employs a 3D vision sensor to implement a smart dispensing application and perform online process quality control of large parts assembly. The aim of the system is to provide execution autonomy for a robotic manipulator that is responsible for glue dispensing on different types of aluminium profiles of varying cross-sections, length, and associated glue patterns. By using an eye-in-hand RGB-D camera and employing deep learning methods (Yolov3 algorithm), the framework recognizes the type of profile that is going to be processed, and depth information is used to extract the profile’s points (starting-ending points) on which glue must be dispensed. Point Cloud manipulation (DBSCAN clustering, plane segmentation) and 2D image-based processing were tested during the quality inspection, in order to validate the success and accuracy of the process (detection of glue discontinuities, calculation of the glue volume level). The system has been deployed on a case study involving the assembly of components on buses and the results indicate that a robust recognition and adhesive application can be obtained throughout each operation cycle.