Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Constrained Consensus and Inequality-constrained Distributed Optimization with Guaranteed Differential Privacy and Accurate Convergence

Yongqiang Wang, Angelia Nedic

TL;DR

This work tackles distributed optimization with a shared inequality constraint $\sum_{i=1}^m g_i(x_i)\le 0$ under $\epsilon$-differential privacy, and develops a co-design of DP-noise injection with inter-agent updates to guarantee convergence to a global optimum without requiring strict convexity of the Lagrangian or separability of the objective.The authors introduce two key constructs: (i) Algorithm 1, a differentially-private constrained consensus that attenuates persistent DP noise via a weakening factor $\chi^k$ while preserving convergence, and (ii) Algorithm 2, a fully distributed constrained optimization scheme with private communications that converges almost surely to an optimal solution and provides $\epsilon$-DP for both cost and constraint functions.Using primal-dual perturbation methods, they show convergence to saddle points of a modified dual problem under non-strict convexity and non-separable objectives, and prove finite cumulative privacy budgets under suitable decay of the noise and step sequences. The results are supported by a demand-response simulation in smart grids, demonstrating practical privacy-utility tradeoffs and robustness to privacy-driven perturbations.

Abstract

We address differential privacy for fully distributed optimization subject to a shared inequality constraint. By co-designing the distributed optimization mechanism and the differential-privacy noise injection mechanism, we propose the first distributed constrained optimization algorithm that can ensure both provable convergence to a global optimal solution and rigorous $ε$-differential privacy, even when the number of iterations tends to infinity. Our approach does not require the Lagrangian function to be strictly convex/concave, and allows the global objective function to be non-separable. As a byproduct of the co-design, we also propose a new constrained consensus algorithm that can achieve rigorous $ε$-differential privacy while maintaining accurate convergence, which, to our knowledge, has not been achieved before. Numerical simulation results on a demand response control problem in smart grid confirm the effectiveness of the proposed approach.

Robust Constrained Consensus and Inequality-constrained Distributed Optimization with Guaranteed Differential Privacy and Accurate Convergence

TL;DR

This work tackles distributed optimization with a shared inequality constraint $\sum_{i=1}^m g_i(x_i)\le 0$ under $\epsilon$-differential privacy, and develops a co-design of DP-noise injection with inter-agent updates to guarantee convergence to a global optimum without requiring strict convexity of the Lagrangian or separability of the objective.The authors introduce two key constructs: (i) Algorithm 1, a differentially-private constrained consensus that attenuates persistent DP noise via a weakening factor $\chi^k$ while preserving convergence, and (ii) Algorithm 2, a fully distributed constrained optimization scheme with private communications that converges almost surely to an optimal solution and provides $\epsilon$-DP for both cost and constraint functions.Using primal-dual perturbation methods, they show convergence to saddle points of a modified dual problem under non-strict convexity and non-separable objectives, and prove finite cumulative privacy budgets under suitable decay of the noise and step sequences. The results are supported by a demand-response simulation in smart grids, demonstrating practical privacy-utility tradeoffs and robustness to privacy-driven perturbations.

Abstract

We address differential privacy for fully distributed optimization subject to a shared inequality constraint. By co-designing the distributed optimization mechanism and the differential-privacy noise injection mechanism, we propose the first distributed constrained optimization algorithm that can ensure both provable convergence to a global optimal solution and rigorous -differential privacy, even when the number of iterations tends to infinity. Our approach does not require the Lagrangian function to be strictly convex/concave, and allows the global objective function to be non-separable. As a byproduct of the co-design, we also propose a new constrained consensus algorithm that can achieve rigorous -differential privacy while maintaining accurate convergence, which, to our knowledge, has not been achieved before. Numerical simulation results on a demand response control problem in smart grid confirm the effectiveness of the proposed approach.
Paper Structure (14 sections, 17 theorems, 89 equations, 1 figure, 1 table)

This paper contains 14 sections, 17 theorems, 89 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Problem Formulation and Preliminaries
  3. Problem Formulation
  4. Primal-Dual Method
  5. Preliminaries
  6. On Differential Privacy
  7. Differentially-private constrained consensus
  8. Convergence Analysis
  9. Differential-Privacy Analysis
  10. Differentially-private distributed constrained optimization
  11. Convergence to an Optimal Solution
  12. Algorithm 2 Ensures $\epsilon$-Differential Privacy
  13. Numerical Simulations
  14. Conclusions

Key Result

Lemma 1

Let $\{v^k\}$, $\{u^k\}$, $\{a^k \}$, and $\{b^k\}$ be random nonnegative scalar sequences such that $\sum_{k=0}^\infty { a^k}<\infty$ and $\sum_{k=0}^\infty {b^k}<\infty$a.s. and where $\mathcal{F}^k=\{v^\ell,u^\ell,{a^\ell},{b^\ell};\, 0\le \ell\le k\}$. Then $\sum_{k=0}^{\infty}u^k<\infty$ and $\lim_{k\to\infty}v^k=v$ hold a.s. for a random variable $v\geq 0$.

Figures (1)

  • Figure 1: Comparison of Algorithm 2 with the existing distributed constrained optimization algorithm by Chang et al. in chang2014distributed (under the same noise level, labeled as PDP) and the differentially private version of the algorithm by Chang et al. in chang2014distributed (using the DP design in Huang et al. in huang2015differentially under the same cumulative privacy budget $\epsilon$, labeled as PDOP).

Theorems & Definitions (43)

  • Lemma 1: polyak87, Lemma 11, page 50
  • Lemma 2
  • Definition 1
  • Remark 1
  • Remark 2
  • Definition 2
  • Remark 3
  • Lemma 3
  • Lemma 4
  • Theorem 1
  • ...and 33 more