Placing Workers at the Center – Evaluating a Human-Centered Digital Assistance System in Automotive Assembly

The practical application of human-centered design in industrial settings, with assembly line workers, still remains insufficiently explored, especially in the automotive industry. However, their involvement during the system development and evaluation is key to creating solutions that are both practical and meet the real-world needs of the end users. This paper presents the prototype evaluation of an applied human-centered design process aimed at creating a digital assistance system for assembly operators in the automotive industry. Following an iterative development process described in our previous papers, which centered on the workers and their needs, we conducted a structured usability evaluation to identify any issues in practical application, assess the usability of the developed prototype, and collect feedback from the workers. To achieve these goals, we adopted a mixed-methods approach, which included three evaluation measures: First, task-based user tests were conducted using realistic tasks from the assembly line. During this, observers documented all issues encountered and the user’s behavior. Secondly, the users completed the System Usability Scale (SUS), a short questionnaire to provide a standardized measure of usability. Thirdly, brief semi-structured interviews were held to assess the users’ perception of the prototype, including their feedback and suggestions for improvement. Our findings from the user tests, SUS, and interview data reveal a positive evaluation of the system and a good reception among workers. However, the evaluation also identified usability issues that will be addressed in the next development cycle of the prototype to further align it with the workers and the practical requirements of the workplace. Thus, this study demonstrates the importance and impact of active worker participation in creating digital assistance systems that are intuitive, meet user demands, integrate well into production environments, and are more readily accepted by the users.