Estimation Network Design framework for efficient distributed optimization

TL;DR

The paper addresses distributed optimization with partially coupled objectives by introducing Estimation Network Design (END), a graph-theoretic framework that allocates and fuses component-wise estimates to exploit problem sparsity. END unifies and extends multiple algorithms—dual methods, ABC, AugDGM, and Push-Sum DGD—by allowing problem-dependent design of estimate and communication graphs, reducing memory and communication while preserving convergence guarantees. Through theory and sensor-network simulations, END demonstrates substantial reductions in communication costs (often exceeding 90–99%) and scalable performance without laborious per-instance convergence analyses. The approach is particularly advantageous for repeated or time-varying problems (e.g., distributed estimation, MPC), where a one-time design cost yields long-term iterative gains and flexibility in directed/time-varying networks.

Abstract

Distributed decision problems features a group of agents that can only communicate over a peer-to-peer network, without a central memory. In applications such as network control and data ranking, each agent is only affected by a small portion of the decision vector: this sparsity is typically ignored in distributed algorithms, while it could be leveraged to improve efficiency and scalability. To address this issue, our recent paper introduces Estimation Network Design (END), a graph theoretical language for the analysis and design of distributed iterations. END algorithms can be tuned to exploit the sparsity of specific problem instances, reducing communication overhead and minimizing redundancy, yet without requiring case-by-case convergence analysis. In this paper, we showcase the flexility of END in the context of distributed optimization. In particular, we study the sparsity-aware version of many established methods, including ADMM, AugDGM and Push-Sum DGD. Simulations on an estimation problem in sensor networks demonstrate that END algorithms can boost convergence speed and greatly reduce the communication and memory cost.

Figures (4)

• Figure 1: (a) A simple example of design from Bianchi_minG_TCNS_2023. On the left, the given communication and interference graphs, with $\mathcal{I} = \{1,2,3,4,5\}$ and $\mathcal{P} = \{1,2\}$. On the right a possible choice for the design graphs and the corresponding estimate graphs. (b) We focus on the design of ${\mathcal{G}}^{ \text{D}}_{1}$. The given efficiency specification is to minimize the number of copies of $y_1$ (i.e., the number of nodes in ${\mathcal{G}}^{ \text{D}}_{1}$), but provided that ${\mathcal{G}}^{ \text{D}}_{1}$ is connected and \ref{['asm:consistency']} is met. Note that agent $2$ has to estimate $y_1$ (i.e., $1 \in \mathcal{N}{~\!\!}^{ \text{E}}({2})$), even though agent $2$ is not directly affected by $y_1$ (i.e., $1 \notin \mathcal{N}{~\!\!}^{ \text{I}}({2})$): otherwise, the information could not travel between nodes $1$ and $3$, which are not communication neighbors. In general, a solution to this design problem can be obtained by solving an Unweighted Steiner Tree problem Chalermsook:Distributed:Steiner:2005, for which distributed off-the-shelf algorithms are available Chalermsook:Distributed:Steiner:2005.
• Figure 2: Distribution of sources (red) and sensors (blue). Sensors in the red circle receive signal from source $p$. Sensors in the blue circle can receive data by (but not necessarily send to) sensor $i$.
• Figure 3: Linear regression via algorithm \ref{['eq:DGD_END']}, for different values of the minimum sensor communication radius $r_\textrm{c}^{\min}$ and stopping criterion $\mathfrak V(\boldsymbol{y}) \leq 10^{-2}$ (bottom), and the trajectories obtained with $r_\textrm{c}^{\min} = 0.1$ (top). A larger $r_\textrm{c}^{\min}$ induces a denser graph $\mathcal{G}^{ \text{C}}$.
• Figure 4: LASSO via algorithm \ref{['eq:DGD_END']}, and different source ranges $r_{\textrm{s}}$.

Theorems & Definitions (11)

• Proposition 1
• Example 1: ADMM
• Theorem 1
• Example 2: AugDGM
• Corollary 1
• Example 3: Choosing time-varying design graphs
• Theorem 2
• Lemma 1
• proof
• Lemma 2