Local explanations, global tensions: the role of explainable AI in reshaping production collaboration

As Artificial Intelligence (AI) becomes more integrated into production processes, it is essential that AI-based tools are transparent and easy to understand for end users such as production planners and machine operators. Yet, many existing eXplainable AI (XAI) approaches are too complex for this target group. In this paper, we examine which forms of human–AI interaction are preferred in manufacturing settings to support end users in better understanding and interpreting AI-generated recommendations. Based on a design case study, we identify user requirements for a more accessible interface and develop a high-fidelity prototype that is qualitatively evaluated through semi-structured interviews. Results support a clear preference for local explanations, while global ones often cause confusion and lack adaptability. Furthermore, while XAI can increase transparency, upper management sometimes withholds information from shop floor workers to maintain existing power structures. This highlights how XAI can influence workplace dynamics, potentially shifting responsibilities and decision-making authority. These findings underline the need to align AI explanations with both the informational needs of production employees and the strategic concerns of management. Doing so can support more effective and context-sensitive implementation of AI systems in production planning.