Classification and Treatment Learning with Constraints via Composite Heaviside Optimization: a Progressive MIP Method
Yue Fang, Junyi Liu, Jong-Shi Pang
TL;DR
This work addresses constrained learning for multiclass classification and multi-action treatment by formulating the problem as Heaviside composite optimization (HSCOP). The authors introduce a progressive mixed-integer programming (PIP) method that solves a sequence of reduced-size MIPs, linking continuous optimization with integer variables to obtain locally optimal solutions with strong practical performance. The approach supports affine policy classes, margins, and a least-residual slack for feasibility under a Gini-based income fairness constraint, with theoretical guarantees tying epi-stationarity to local optimality under suitable conditions. Empirical results on synthetic data demonstrate that PIP achieves comparable welfare to full MIP while delivering substantial computational speedups, particularly as the problem size grows and the Gini constraint tightens, highlighting its potential for scalable constrained learning in practice.
Abstract
This paper proposes a Heaviside composite optimization approach and presents a progressive (mixed) integer programming (PIP) method for solving multi-class classification and multi-action treatment problems with constraints. A Heaviside composite function is a composite of a Heaviside function (i.e., the indicator function of either the open $( \, 0,\infty )$ or closed $[ \, 0,\infty \, )$ interval) with a possibly nondifferentiable function. Modeling-wise, we show how Heaviside composite optimization provides a unified formulation for learning the optimal multi-class classification and multi-action treatment rules, subject to rule-dependent constraints stipulating a variety of domain restrictions. A Heaviside composite function has an equivalent discrete formulation, and the resulting optimization problem can in principle be solved by integer programming (IP) methods. Nevertheless, for constrained learning problems with large data sets, a straightforward application of off-the-shelf IP solvers is usually ineffective in achieving global optimality. To alleviate such a computational burden, our major contribution is the proposal of the PIP method by leveraging the effectiveness of state-of-the-art IP solvers for problems of modest sizes. We provide the theoretical advantage of the PIP method with the connection to continuous optimization and show that the computed solution is locally optimal for a broad class of Heaviside composite optimization problems. The numerical performance of the PIP method is demonstrated by extensive computational experimentation.