The metrics trap: how technical sophistication masks social harm in urban AI systems

This study examines the deployment of artificial intelligence in urban systems through a multiple case study analysis of 28 implementations selected from 157 AI deployments across six domains (2015–2024). We reveal how cities have fallen into a “metrics trap”, pursuing technical accuracy that fails to achieve policy goals. Our analysis documents three critical patterns: (1) an “accuracy illusion” where impressive performance metrics mask fundamental failures, exemplified by ShotSpotter’s 97% acoustic accuracy yielding only 9.1% crime-fighting effectiveness; (2) discriminatory feedback loops that transform historical bias into computational destiny, affecting millions through predictive policing and housing algorithms; and (3) successful community resistance movements from Toronto to Detroit proving technological determinism is a myth. Cases including Amsterdam’s algorithm register, Seoul’s participatory waste management, and NYC Health + Hospitals’ bias mitigation reveal varied approaches to democratic AI governance. These findings suggest cities can pursue both technical capability and democratic accountability.