FedSGM: A Unified Framework for Constraint Aware, Bidirectionally Compressed, Multi-Step Federated Optimization

TL;DR

This work tackles constrained federated learning where a global objective $f(w)=\tfrac{1}{n}\sum_j f_j(w)$ is minimized subject to a global constraint $g(w)\le 0$, under communication constraints and partial client participation. It introduces FedSGM, a duality-free, projection-free, primal-only switching gradient framework that enables bi-directional error-feedback compression and supports multi-step local updates with hard or soft switching to enforce feasibility. The authors provide convergence guarantees for the averaged iterate with the canonical $\mathcal{O}(1/\sqrt{T})$ rate, plus high-probability bounds that decouple optimization progress from sampling noise; they further show how soft switching stabilizes updates near feasibility boundaries. Empirically, FedSGM demonstrates reliable convergence on Neyman-Pearson classification and constrained CMDP tasks, validating the theoretical results and illustrating robustness to various degrees of local computation, participation, and communication compression. Overall, the paper establishes a theoretically grounded, scalable framework for constrained Federated Learning that unifies feasibility, compression, local updates, and participation.

Abstract

We introduce FedSGM, a unified framework for federated constrained optimization that addresses four major challenges in federated learning (FL): functional constraints, communication bottlenecks, local updates, and partial client participation. Building on the switching gradient method, FedSGM provides projection-free, primal-only updates, avoiding expensive dual-variable tuning or inner solvers. To handle communication limits, FedSGM incorporates bi-directional error feedback, correcting the bias introduced by compression while explicitly understanding the interaction between compression noise and multi-step local updates. We derive convergence guarantees showing that the averaged iterate achieves the canonical $\boldsymbol{\mathcal{O}}(1/\sqrt{T})$ rate, with additional high-probability bounds that decouple optimization progress from sampling noise due to partial participation. Additionally, we introduce a soft switching version of FedSGM to stabilize updates near the feasibility boundary. To our knowledge, FedSGM is the first framework to unify functional constraints, compression, multiple local updates, and partial client participation, establishing a theoretically grounded foundation for constrained federated learning. Finally, we validate the theoretical guarantees of FedSGM via experimentation on Neyman-Pearson classification and constrained Markov decision process (CMDP) tasks.

Figures (7)

• Figure 1: NP classification: Progress per round with hard and soft switching.
• Figure 2: Top: Effect of local updates. Middle: Effect of partial participation. Bottom: Effect of compression on objective and feasibility for hard (left) and soft (right) switching.
• Figure 3: CMDP: Episodic reward vs. Cost. Dotted: centralized, solid: federated with $m/n=0.7$ and Top$-K$, $K/d=0.5$. Red dotted line: safety budget of $30$.
• Figure 4: Effect of partial participation on episodic reward and cost. Total clients $n=10$, no compression. Red dotted line: safety budget of $30$.
• Figure 5: NP classification: Comparison of soft switching parameter in FedSGM. Solid lines and shaded areas indicate the mean and standard deviation over 3 runs with different random seeds respectively. Red dotted line indicates the constraint violation threshold $\epsilon$.

Theorems & Definitions (43)

• Theorem 1
• Remark 1
• Theorem 2
• Proposition 1
• proof : Proof sketch
• lemma 1
• proof
• lemma 2
• proof
• lemma 3