Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decentralized Optimization with Mixed Affine Constraints

Demyan Yarmoshik, Nhat Trung Nguyen, Alexander Rogozin, Alexander Gasnikov

TL;DR

This paper proposes a unified framework for decentralized convex optimization under mixed affine constraints that combine coupled, local, and shared-variable restrictions. It develops optimal first-order methods—principally APAPC and Gradient Sliding—with Chebyshev preconditioning to achieve tight iteration and communication bounds in the smooth strongly convex setting, and near-optimal results in non-strongly convex and nonsmooth regimes. The analysis introduces new conditioning measures (e.g., mixed condition numbers) and shows how the joint structure of the constraints and the network topology governs performance, unifying horizontal and vertical federated learning, distributed multi-task learning, and control applications. The results provide practical guidance for designing efficient decentralized algorithms that balance local computations, communications, and constraint handling, with explicit complexity bounds across all problem variants.

Abstract

This paper considers decentralized optimization of convex functions with mixed affine equality constraints involving both local and global variables. Constraints on global variables may vary across different nodes in the network, while local variables are subject to coupled and node-specific constraints. Such problem formulations arise in machine learning applications, including federated learning and multi-task learning, as well as in resource allocation and distributed control. We analyze this problem under smooth and non-smooth assumptions, considering both strongly convex and general convex objective functions. Our main contribution is an optimal algorithm for the smooth, strongly convex regime, whose convergence rate matches established lower complexity bounds. We further provide near-optimal methods for the remaining cases.

Decentralized Optimization with Mixed Affine Constraints

TL;DR

This paper proposes a unified framework for decentralized convex optimization under mixed affine constraints that combine coupled, local, and shared-variable restrictions. It develops optimal first-order methods—principally APAPC and Gradient Sliding—with Chebyshev preconditioning to achieve tight iteration and communication bounds in the smooth strongly convex setting, and near-optimal results in non-strongly convex and nonsmooth regimes. The analysis introduces new conditioning measures (e.g., mixed condition numbers) and shows how the joint structure of the constraints and the network topology governs performance, unifying horizontal and vertical federated learning, distributed multi-task learning, and control applications. The results provide practical guidance for designing efficient decentralized algorithms that balance local computations, communications, and constraint handling, with explicit complexity bounds across all problem variants.

Abstract

This paper considers decentralized optimization of convex functions with mixed affine equality constraints involving both local and global variables. Constraints on global variables may vary across different nodes in the network, while local variables are subject to coupled and node-specific constraints. Such problem formulations arise in machine learning applications, including federated learning and multi-task learning, as well as in resource allocation and distributed control. We analyze this problem under smooth and non-smooth assumptions, considering both strongly convex and general convex objective functions. Our main contribution is an optimal algorithm for the smooth, strongly convex regime, whose convergence rate matches established lower complexity bounds. We further provide near-optimal methods for the remaining cases.
Paper Structure (62 sections, 22 theorems, 109 equations, 4 tables, 3 algorithms)

This paper contains 62 sections, 22 theorems, 109 equations, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Horizontal Federated Learning (HFL, or Consensus Optimization).
  3. Vertical Federated Learning (VFL)
  4. Related Work.
  5. Our Contributions.
  6. Notation.
  7. Paper Organization.
  8. Optimization with Affine Constraints
  9. Assumptions on the Objective Function
  10. Smooth Objectives
  11. Non-Smooth Objectives
  12. Chebyshev Acceleration
  13. Existing Results in Decentralized Optimization with Affine Constraints
  14. Notations and Assumptions
  15. Consensus Optimization
  16. ...and 47 more sections

Key Result

Theorem 2.4

Let $G(u)$ satisfy Assumptions assum:strongly_convex and assum:smooth with $0 < \mu < L$. Let $\left\| u^0 - u^* \right\|^2\leq R^2$ and introduce accuracy $\varepsilon > 0$. 1. Strongly convex case $\mu > 0$salim2022optimal. There exists a set of parameters for alg:apapc such that after $N = O\left

Theorems & Definitions (34)

  • Theorem 2.4
  • Theorem 2.5: lan2020communication
  • Definition 3.4
  • Remark 3.5
  • Remark 3.6
  • Theorem 4.1: new, Appendix \ref{['app:nonind_local_lower']}
  • Definition 4.2
  • Remark 4.3
  • Lemma 4.4: new, Appendix \ref{['app:mixed_coupled_with_local_spectrum']}
  • Theorem 4.5: new, Appendix \ref{['app:mixed_coupled_with_local_lower']}
  • ...and 24 more