A common challenge in medicine is the allocation of limited resources. Prediction models can assist in making resource allocation decisions, but they may also exhibit unfairness. We introduce a post-processing algorithm tailored to this scenario and compare its performance to modified pre-processing and in-processing algorithms. We developed predictors for in-hospital mortality, utilizing ICU data from two datasets. We then applied a post-processing algorithm we developed to enforce equal opportunity while adhering to a limited resources constraint and compared it to modified existing pre-processing and in-processing algorithms. The results were evaluated in terms of fairness and predictive performance, and the usability of the three algorithms was compared. All algorithms showed substantial improvement in the "equal opportunity" metric, presenting an average decrease of 52% in the span of sensitivity values across subgroups, and none of the algorithms consistently outperformed the others. In some cases enforcing fairness reduced predictive performance, with an average decrease of 4% in sensitivity and 3% in positive predictive value. However, there were usability differences between the algorithms: the pre-processing and post-processing algorithms preserve the numerical risk predictions, and only the post-processing algorithm does not require re-training to change the size of the intervention group. Our results demonstrate that all three methods enhance model fairness, with no single approach consistently outperforming the others. All three methods also achieve similar overall predictive performance, which in some cases is reduced compared to the base model. However, our post-processing algorithm offers practical advantages in usability compared to the alternatives.
Stern et al. (Thu,) studied this question.