When Algorithms Meet Institutions: Evaluation Ecologies in Public Sector Machine Learning

This article introduces the concept of “evaluation ecologies” to theorize how machine learning (ML) systems are assessed in public sector contexts. Drawing on Science and Technology Studies (STS) and critical AI scholarship, and supported by case studies of ML deployments in Danish educational institutions and Dutch psychiatric clinics, we show that ML evaluation is not a discrete technical procedure but an ongoing, contested process in which multiple registers of judgment interact. Building on Halpern and Mitchell’s (2023) account of experimental governance and Amoore’s (2020) analysis of cloud ethics, the notion of evaluation ecologies captures how technical metrics, professional discretion, legal frameworks, ethical norms, and political objectives coexist and compete in shaping algorithmic outcomes. The Danish case demonstrates how legal and ethical registers can terminate an algorithm despite technical robustness, while the Dutch case shows how perceptions of operational utility and policy alignment may override professional skepticism about algorithmic predictions. Together, these cases reveal that ML evaluation in public institutions is fundamentally relational and context-dependent, structured by power, expertise, and accountability rather than by performance metrics alone. By foregrounding evaluation as an ecological and political process, the article contributes to debates on algorithmic governance by showing how local contexts and professional judgment remain decisive in the age of public-sector AI.