DIF-PP: Threshold Optimization Informed by IRT Models for Group Fairness in Machine Learning

Machine learning models risk reproducing social biases, so algorithmic decision-making must incorporate principles that prevent discrimination. Post-processing methods, such as threshold optimization, can support this goal, but striking an appropriate trade-off between predictive performance and group equity metrics remains challenging. The recent application of Differential Item Functioning (DIF) in model selection sheds light on its potential as a promising yet unexplored approach to threshold tuning in more unbiased machine learning systems. Building on this premise, this work introduces DIF-PP, a fairness-aware post-processing method that draws on concepts from Item Response Theory (IRT) and DIF for optimizing decision boundaries. DIF-PP represents these thresholds as test items, derives classification characteristic curves through IRT, and uses DIF to identify the most impartial cutoff point. Experimental results with 18 datasets show that DIF-PP consistently outperforms existing methods when we analyze group fairness metrics and predictive performance simultaneously. By combining IRT and DIF, our proposal effectively mitigates discriminatory effects in binary classification, marking a significant advance toward the development of responsible artificial intelligence solutions.