From foundation models to the factory floor: Evaluating automated annotation for disassembly tasks supported by mixed reality

The growing volume of electronic waste poses a critical environmental burden and a significant challenge for resource circularity. It makes the efficient recovery of materials both a necessity and an opportunity, as the disassembly of these devices facilitates the recovery of hazardous and valuable materials, such as batteries and motors. The recovery process is usually constrained by the high variability in design, component layout, and internal structure across products in the same family. This is compounded by the actual state of the device, as parts can be missing or damaged. Such variability renders full automation intractable, compelling a reliance on manual or semi-automated operations. Technologies such as Mixed Reality can support operators by providing contextual instructions, such as the identification and localization of objects of interest. In complex products, the precision of the localization is important to correctly identify the target object for a given step and avoid identification errors. In many cases, product components are very close to each other and a localization error can lead to an identification mismatch. Although Artificial Intelligence-based 6D pose estimation models are key enablers for this functionality, they require large, accurately annotated training datasets, a labor-intensive bottleneck in industrial contexts. This paper proposes automating dataset annotation using foundation models. This automated approach, however, introduces inherent uncertainties regarding label precision that must be characterized. The primary contribution of this work is a rigorous, in-depth analysis of the sources of error propagating from automated dataset annotation. We evaluate the impact of these inaccuracies on model performance within the specific context of a real-world industrial disassembly application.