Ethical guidelines for machine learning competitions

Machine learning and artificial intelligence competitions are commonly used as platforms for developing machine learning systems, yet the ethical considerations associated with these systems are rarely made explicit. In this paper, we explore the ethical considerations relevant to competitions hosted on online platforms such as Kaggle, AIcrowd, and DrivenData, including issues related to algorithmic bias, data privacy, transparency, explainability, agency, and compliance. Through an analysis of over 800 competitions across three platforms, we find that discussion of ethical considerations in the competitions’ documentation is often limited in scope and inconsistently addressed across competition types. Based on these results, we propose the inclusion of a dedicated section on ethical considerations and risks within competition documentation. This section would describe potential harms associated with participation, enabling participants to make informed decisions about their involvement. Additionally, this section may support greater awareness of ethical considerations in machine learning and artificial intelligence.