Machine learning for environmental justice: Dissecting an algorithmic approach to predict drinking water quality in California

The potential for machine learning to answer questions of environmental science, monitoring, and regulatory enforcement is evident, but there is cause for concern regarding potential embedded bias: algorithms can codify discrimination and exacerbate systematic gaps. This paper, organized into two halves, underscores the importance of vetting algorithms for bias when used for questions of environmental science and justice. In the first half, we present a case study of using machine learning for environmental justice-motivated research: prediction of drinking water quality. While performance varied across models and contaminants, some performed well. Multiple models had overall accuracy rates at or above 90 % and F2 scores above 0.60 on their respective test sets. In the second half, we dissect this algorithmic approach to examine how modeling decisions affect modeling outcomes - and not only how these decisions change whether the model is correct or incorrect, but for whom. We find that multiple decision points in the modeling process can lead to different predictive outcomes. More importantly, we find that these choices can result in significant differences in demographic characteristics of false negatives. We conclude by proposing a set of practices for researchers and policy makers to follow (and improve upon) when applying machine learning to questions of environmental science, management, and justice.