‘Auditing for equity’: Desiderata for auditing of biases in current data-driven health systems

In the discourse on advancing health equity, there is an opportunity for technologists to work with scholars of public health and allied disciplines. Much of current discussion revolves around ‘bigger and better’ technologies, such as large language models (LLMs). However, from the perspective of applied digital ethics, questions such as ‘do existing technologies fail to do justice to individuals?’ need to be asked. To wit, what are “social, economic, and environmental factors perpetuating health disparities” (Xyrichis et al., 2024) captured by extant public-health-related AI and data-driven systems? This piece will provide a primer on data bias and error in public-health-related systems, and recognize how such issues perpetuate inequalities in health outcomes. This includes an awareness of factors such as spurious correlations, stereotyping, and non-representative data, to name a few. The paper will then recommend several actionable auditing techniques in order to systematically document such public-health-related systems, such as Datasheets for Datasets and Model Cards (Gebru et al., 2021; Mitchell et al., 2019), as a starting point for long-term improvements. Common issues such as the myriad of auditing frameworks to select from, interdisciplinary silos and barriers to change, as well as concerns of ‘ethics-washing’ will be discussed. Overall, this paper aims to be a conversation-starter to bridge the gap between technology and public health, to enable practitioners to ‘look inward’ to identify how “data science and allied methods can strengthen public health research” (Goldsmith et al., 2021).