Stochastic Gradient Descent-like relaxation is equivalent to Metropolis dynamics in discrete optimization and inference problems

TL;DR

This work addresses whether an SGD-like relaxation on discrete optimization problems is fundamentally different from Metropolis MC dynamics. By analyzing the planted and random $q$-coloring problem, the authors show that SGD-like dynamics with mini-batch size $B$ closely emulate Metropolis MC dynamics at an effective temperature $T$ that is a function of $B$, in both equilibrium and out-of-equilibrium regimes. Although SGD-like does not strictly satisfy detailed balance, an averaged condition yields an effective mapping that accounts for the observed dynamical equivalence, enabling the use of Monte Carlo results to guide mini-batch optimization and improve recovery performance. The findings suggest that fluctuations introduced by mini-batching play a role analogous to thermal fluctuations and open avenues to transfer MC theory to SGD-like algorithms, with potential extensions to continuous and dense problems.

Abstract

Is Stochastic Gradient Descent (SGD) substantially different from Metropolis Monte Carlo dynamics? This is a fundamental question at the time of understanding the most used training algorithm in the field of Machine Learning, but it received no answer until now. Here we show that in discrete optimization and inference problems, the dynamics of an SGD-like algorithm resemble very closely that of Metropolis Monte Carlo with a properly chosen temperature, which depends on the mini-batch size. This quantitative matching holds both at equilibrium and in the out-of-equilibrium regime, despite the two algorithms having fundamental differences (e.g.\ SGD does not satisfy detailed balance). Such equivalence allows us to use results about performances and limits of Monte Carlo algorithms to optimize the mini-batch size in the SGD-like algorithm and make it efficient at recovering the signal in hard inference problems.

Figures (9)

• Figure 1: Intensive energy reached by the SGD-like algorithm with three different values of $B$ as a function of time, for a single system of size $N=10^4$ and mean connectivity $c=19$. For $B=0.86$ the algorithm ends in a paramagnetic state, for $B=0.95$ it ends in some spurious glassy states, while for $B=0.9$ the algorithm manages to find the low-energy planted state. Inset: overlap as defined in Eq. (\ref{['eq:overlap']}) between the planted state and the configurations visited at time $t$ by the SGD-like algorithm with the same values of $B$ as in the main Figure.
• Figure 2: Average energy $\overline{e}$ mediated over 280 samples for three different sizes $N=10^3, 10^4, 10^5$ (from left to right), for MC and SGD-like algorithms. Left: In MC at a temperature $T=0.49$ and SGD-like algorithm with a mini-batch parameter $B=0.92$ the energy relaxes in a very similar way and the average nucleation times coincide. Right: as in the left panel, but with MC run at $T=0.47$ and an SGD-like algorithm with $B=0.928$.
• Figure 3: Averaged nucleation time $\overline{t_\text{nucl}}$ (left) and the corresponding standard deviation $\sigma_{t_\text{nucl}}$ (right) for the same matching $(T,B)$ pairs used in Fig. \ref{['Fig:energy_recovery']}. Both $\overline{t_\text{nucl}}$ and $\sigma_{t_\text{nucl}}$ are very similar for the two algorithms and grow almost linearly with the problem size.
• Figure 4: Data for a single system of size $N=10^5$ and $c=19$. Left: Intensive energy $e$ and two-time correlations $c(t,t_w)$ at different values of $t_w$ for MC algorithm at $T=0.635$ and a SGD-like algorithm with $B=0.87$. Values of $T$ and $B$ have been matched equating the plateau energy, and in turn, make the entire relaxation process of the two algorithms very similar. Right: As in the left panel, for the MC algorithm at $T=0.4$ (lines) and the SGD-like algorithm with $B=0.95$ (points). Both algorithms end in a low-temperature, aging regime, signaled by $c(t,t_w)$ depending on $t_w$.
• Figure 5: Left:$G(B,s,u)$ as a function of $B$, for all possible values of $s$ and $u$ for $c=19$: the fact that it depends on $s$ and $u$ implies the failure of detailed balance (see text for more details) Right: The blue dashed line corresponds to $\left(\overline{G(B,s,u)}\right)^{-1}$, the blue full lines correspond to the maximum and minimum values of $(G(B,s,u))^{-1}$ over the possible choices of $s$ and $u$. Crosses and circles correspond to $(B, T)$ pairs obtained in previous sections through the matching condition on the plateau energy, which in turn make the entire dynamics of the two algorithms very similar. Horizontal dashed lines correspond to $T_\text{\tiny PM}$ and $T_\text{glassy}$, the only recovery region for the MC algorithm being $T_\text{glassy}<T<T_\text{\tiny PM}$.