The Taming of the Bias: Measuring and Mitigating Category-Aware Gender Bias in Recommendations

Recommenders are a vital part of our daily lives. We rely on them for tasks such as discovering new music and connecting job seekers with relevant opportunities. Given their crucial role, we must ensure recommendations are free from societal stereotypes. Therefore, evaluating and mitigating biases in them is essential. Previous work used to evaluate the fairness of recommended items fails to capture certain nuances, by mostly focusing on comparing performance metrics for different sensitive groups. In this paper, we introduce a set of comprehensive metrics for evaluating gender bias in recommendations. We highlight the importance of evaluating fairness at a granular level, which can be achieved with our set of metrics, using item categories, like genres for movies. Furthermore, we show that adding a category-aware fairness metric as a regularization term to the main recommendation loss during training can help effectively minimize bias in the models’ output. We experiment on three real-world datasets, using five baseline models, to show the effectiveness of our metrics in evaluating gender bias. Additionally, we use two of our metrics as separate regularization terms, to demonstrate how incorporating them significantly reduces the bias in recommendations for different categories without substantial degradation in overall recommendation performance. The link to the code and datasets are provided here: GitHub repository