Local Second-Order Limit Dynamics of the Alternating Direction Method of Multipliers for Semidefinite Programming
Shucheng Kang, Heng Yang
TL;DR
This paper tackles slow convergence of ADMM when solving large-scale semidefinite programs with multiple KKT points by developing a local, second-order limit-dynamics framework around an arbitrary KKT point. Central to the approach is a refined parabolic second-order directional derivative of the PSD projection, from which a local limit map is derived to describe persistent drift after transients are filtered out; the analysis reveals a primal–dual decoupling and a cone of directions where first-order updates vanish. The authors establish structural properties of the limit map (kernel, range, continuity) and show how the penalty parameter $\sigma$ influences the dynamics, connecting these findings to fixed points, almost-invariant sets, and microscopic phases. Numerical experiments on Mittelmann SDP instances corroborate the theory, explaining phenomena such as small nonzero angles between iterate differences, infeasibility metrics that are insensitive to $\sigma$, and transient confinement to a low-dimensional subspace, thereby providing practical guidance for algorithm design and parameter tuning. The work offers a physics-informed lens on region-wise transient behavior in first-order methods for SDPs and lays groundwork for accelerated strategies leveraging second-order limit dynamics.
Abstract
The alternating direction method of multipliers (ADMM) is widely used for solving large-scale semidefinite programs (SDPs), yet on instances with multiple primal--dual optimal solution pairs, it often enters prolonged slow-convergence regions where the Karush--Kuhn--Tucker (KKT) residuals nearly stall. To explain and predict the fine-grained dynamical behavior inside these regions, we develop a local second-order limit dynamics framework for ADMM near an arbitrary KKT point -- not necessarily the eventual limit point of the iterates. Assuming the existence of a strictly complementary primal--dual solution pair, we derive a second-order local expansion of the ADMM dynamics by leveraging a refined and simplified variational characterization of the (parabolic) second-order directional derivative of the PSD projection operator. This expansion reveals a closed convex cone of directions along which the local first-order update vanishes, and it induces a second-order limit map that governs the persistent drift after transient effects are filtered out. We characterize fundamental properties of this mapping, including its kernel, range, and continuity. A primal--dual decoupling further yields a clean scaling law for the effect of the penalty parameter in ADMM. We connect these properties to second-order dynamical features of ADMM, including fixed points, almost-invariant sets, and microscopic phases. Three empirical phenomena in slow-convergence regions are then explained or predicted: (i) angles between consecutive iterate differences are small yet nonzero, except for sparse spikes; (ii) primal and dual infeasibilities are insensitive to penalty-parameter updates; and (iii) iterates can be transiently trapped in a low-dimensional subspace for an extended period. Extensive numerical experiments on the Mittelmann dataset corroborate our theoretical predictions.