Contesting Efficacy: Tensions Between Risk and System Efficacy in Facial Analysis Software

Machine learning (ML) applications are frequently trained to make predictions about human characteristics. In the realm of computer vision, where facial analysis tasks like facial classification and facial recognition use visual data to classify attributes about human identity, predictions are often done without any user input at all. However, these systems are repeatedly wrong. They make errors about classification, they propagate social biases, and they constrain complex human identities—like ethnicity and gender—into simple schemas. Contestability and user input are interesting paths forward when considering how to improve classification by facial analysis. However, there are many tradeoffs—technical and ethical—to consider when attempting to embed contestability in computer vision systems. In this position paper, I describe some of the tensions of user autonomy and efficacy in computer vision tasks that need further attention in HCI, ML, and social computing research.