A New Decomposition Paradigm for Graph-structured Nonlinear Programs via Message Passing

TL;DR

This work introduces MP-Jacobi, a decentralized decomposition framework for graph-structured nonlinear programs that merges min-sum message passing on tree clusters with Jacobi-style inter-cluster updates, enabling 1-hop communication and convergence on loopy graphs. The authors establish global linear convergence for strongly convex objectives and provide explicit rate bounds that reveal how partition choice, curvature, and coupling affect scalability. To reduce per-iteration cost, they develop graph-compliant surrogates that preserve convergence while exchanging lightweight vector information, and extend the framework to hypergraphs with a hyperedge-splitting surrogate when overlaps are dense. Theoretical results are complemented by experiments showing consistent improvements over decentralized gradient baselines, demonstrating MP-Jacobi as a scalable primitive for distributed optimization with localized couplings and practical applicability to networked systems.

Abstract

We study finite-sum nonlinear programs whose decision variables interact locally according to a graph or hypergraph. We propose MP-Jacobi (Message Passing-Jacobi), a graph-compliant decentralized framework that couples min-sum message passing with Jacobi block updates. The (hyper)graph is partitioned into tree clusters. At each iteration, agents update in parallel by solving a cluster subproblem whose objective decomposes into (i) an intra-cluster term evaluated by a single min-sum sweep on the cluster tree (cost-to-go messages) and (ii) inter-cluster couplings handled via a Jacobi correction using neighbors' latest iterates. This design uses only single-hop communication and yields a convergent message-passing method on loopy graphs. For strongly convex objectives we establish global linear convergence and explicit rates that quantify how curvature, coupling strength, and the chosen partition affect scalability and provide guidance for clustering. To mitigate the computation and communication cost of exact message updates, we develop graph-compliant surrogates that preserve convergence while reducing per-iteration complexity. We further extend MP-Jacobi to hypergraphs; in heavily overlapping regimes, a surrogate-based hyperedge-splitting scheme restores finite-time intra-cluster message updates and maintains convergence. Experiments validate the theory and show consistent improvements over decentralized gradient baselines.

Figures (6)

• Figure 1: (a) Intra-cluster and inter-cluster messages at node $i$: $\mu^{\star}_{\mathcal{N}^{\mathrm{in}}_i\to i}$ collects contributions from neighbors within $\mathcal{G}_r$ (red nodes) while $\mu^{\star}_{\mathcal{N}^{\mathrm{out}}_i\to i}$ collects those from neighbors outside the cluster (blue nodes). (b) Message propagation within $\mathcal{G}_r$: messages are initialized at the leaves (light red nodes) and propagated along the tree according to the recursion～\ref{['eq:ideal_message_new']}.
• Figure 2: Bird’s-eye view of Algorithm \ref{['alg:main']}. The graph is decomposed into tree clusters (possibly singletons) $(\mathcal{C}_i)_{i=1}^5$. Intra-cluster interactions are handled by min-sum message passing (within the nonsingleton supernodes $\texttt{c}_1$ and $\texttt{c}_2$), whereas inter-cluster interactions are handled by Jacobi-type updates at boundary nodes (singleton clusters) $\texttt{c}_3$ and $\texttt{c}_4$, and $\texttt{c}_5$. At the cluster level, the iterations approximate a block-Jacobi step on the condensed graph.
• Figure 3: Graph with two cliques (degree 6) connected by a long path (length 42). All partitions (different $p$) share the same $D$, $A_{\mathcal{J}}$, $(\mu_r)_{r\in\mathcal{J}}$ and $\kappa$.
• Figure 4: Graph with two cliques connected by a long path (length =20); $A_{\mathcal{J}}$ varies, with constant $p$, $D$, $(\mu_r)_{r\in\mathcal{J}}$ and $\kappa$.
• Figure 5: Graph as in panel (c). In (a), $\kappa$ varies while keeping all the other parameters fixed, whereas in (b) only $A_{\mathcal{J}}$ changes.

Theorems & Definitions (20)

• definition thmcounterdefinition: condensed graph
• proposition thmcounterproposition
• proof
• proposition thmcounterproposition
• lemma thmcounterlemma
• proof
• lemma thmcounterlemma
• proof
• lemma thmcounterlemma
• proof