The Murky Waters of Algorithmic Profiling

The Murky Waters of Algorithmic Profiling: Examining discrimination in the digitalized enforcement of social security policy In the Netherlands, organizations tasked with investigating and sanctioning social security fraud often use algorithmic profiling technologies. Despite the potential benefits, such technologies are not immune to risks, such as discrimination. A lack of clarity in the translation of non-discrimination legislation into practice entails that organizations are tasked with engaging and operationalizing this legislation ex-ante through the identification and mitigation of risks of discrimination. This paper explores the question: How does the use of algorithmic profiling technologies lead to discrimination and can social security organizations mitigate these risks? In order to answer this question, the theoretical framework of the algorithmic lifecycle is used to identify risks of discrimination in the design, implementation, and use of algorithmic profiling technologies. In particular, semi-structured interviews with experts and the Dutch Childcare Benefits Scandal case are used to identify risks of discrimination and examine how they interact to produce discriminatory outcomes in practice. The findings demonstrate that discrimination is the result of a complex interaction of unmitigated risks throughout the algorithmic lifecycle of profiling technologies. This highlights the importance of taking a lifecycle approach to the identification and mitigation of risks of discrimination.