Federated Learning with Convex Global and Local Constraints
Chuan He, Le Peng, Ju Sun
TL;DR
This work addresses federated learning for problems with general convex global and local constraints, a setting with privacy-sensitive distributed data. It develops a proximal augmented Lagrangian framework in which outer iterations solve unconstrained proximal subproblems and inner ADMM-based solvers enforce constraints in a federated manner. The authors establish worst-case iteration complexity results and verify linear convergence for the subproblem solvers under locally Lipschitz gradients, along with clear communication overhead analyses. Numerical experiments on Neyman-Pearson classification and fairness-constrained learning demonstrate that the proposed FL approach achieves feasibility and competitive objective values comparable to centralized proximal-AL methods, validating its practical relevance. The paper thus provides the first general FL methodology with rigorous guarantees for convex global and local constraints, offering a foundation for broader constrained FL applications.
Abstract
In practice, many machine learning (ML) problems come with constraints, and their applied domains involve distributed sensitive data that cannot be shared with others, e.g., in healthcare. Collaborative learning in such practical scenarios entails federated learning (FL) for ML problems with constraints, or FL with constraints for short. Despite the extensive developments of FL techniques in recent years, these techniques only deal with unconstrained FL problems or FL problems with simple constraints that are amenable to easy projections. There is little work dealing with FL problems with general constraints. To fill this gap, we take the first step toward building an algorithmic framework for solving FL problems with general constraints. In particular, we propose a new FL algorithm for constrained ML problems based on the proximal augmented Lagrangian (AL) method. Assuming convex objective and convex constraints plus other mild conditions, we establish the worst-case complexity of the proposed algorithm. Our numerical experiments show the effectiveness of our algorithm in performing Neyman-Pearson classification and fairness-aware learning with nonconvex constraints, in an FL setting.