An automatic system to detect equivalence between iterative algorithms
Shipu Zhao, Laurent Lessard, Madeleine Udell
TL;DR
This work establishes a control-theoretic framework for recognizing when iterative optimization algorithms are equivalent by modeling algorithms as linear systems in feedback with nonlinear oracles and comparing their transfer functions. It introduces two core equivalences—oracle equivalence (identical transfer functions for the same oracle sequence) and shift equivalence (equivalence under cyclic permutations of update steps)—and shows how minimal realizations, state-space transformations, and conjugation of oracles reveal deep connections among classical methods such as Douglas–Rachford, ADMM, Chambolle–Pock, and proximal gradient. The Linnaeus software automates this analysis, translating user-described algorithms into canonical transfer-function representations to detect equivalences, perform symbolic parameter solving, and serve as a literature search engine for optimization mathematics. While powerful for linear-time-invariant settings, the framework also outlines extensions to stochastic, parallel, and nonlinear contexts, highlighting future directions to broaden the reach of mathematical searchability in optimization. Overall, the approach provides principled, automatable insight into the relationships between existing algorithms and the novelty of new proposals, with practical impact on algorithm design and bibliographic navigation.
Abstract
When are two algorithms the same? How can we be sure a recently proposed algorithm is novel, and not a minor twist on an existing method? In this paper, we present a framework for reasoning about equivalence between a broad class of iterative algorithms, with a focus on algorithms designed for convex optimization. We propose several notions of what it means for two algorithms to be equivalent, and provide computationally tractable means to detect equivalence. Our main definition, oracle equivalence, states that two algorithms are equivalent if they result in the same sequence of calls to the function oracles (for suitable initialization). Borrowing from control theory, we use state-space realizations to represent algorithms and characterize algorithm equivalence via transfer functions. Our framework can also identify and characterize some algorithm transformations including permutations of the update equations, repetition of the iteration, and conjugation of some of the function oracles in the algorithm. To support the paper, we have developed a software package named Linnaeus that implements the framework to identify other iterative algorithms that are equivalent to an input algorithm. More broadly, this framework and software advances the goal of making mathematics searchable.