Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals

We show that many machine learning goals, such as improved fairness metrics, be expressed as constraints on the model's predictions, which we call rate. We study the problem of training non-convex models subject to rate constraints (or any non-convex and non-differentiable constraints). the non-convex setting, the standard approach of Lagrange multipliers may. Furthermore, if the constraints are non-differentiable, then one cannot the Lagrangian with gradient-based methods. To solve these issues, we the proxy-Lagrangian formulation. This new formulation leads to an that produces a stochastic classifier by playing a two-player-zero-sum game solving for what we call a semi-coarse correlated, which in turn corresponds to an approximately optimal and feasible to the constrained optimization problem. We then give a procedure shrinks the randomized solution down to one that is a mixture of at mostm+1 deterministic solutions, given m constraints. This culminates in that can solve non-convex constrained optimization problems with non-differentiable and non-convex constraints with theoretical. We provide extensive experimental results enforcing a wide range of goals including different fairness metrics, and other goals on accuracy, , recall, and churn.