The maximum-average subtensor problem: equilibrium and out-of-equilibrium properties

TL;DR

This work studies the Maximum-Average Subtensor ($p$-MAS) problem, a tensor-generalization of MAS, by mapping random tensors onto Ising-like spin models and analyzing both equilibrium and out-of-equilibrium properties in the regime $1\\ll k\\ll N$. The authors derive an equilibrium phase diagram featuring RS and frozen 1-RSB phases, provide analytic thresholds for the maximum subtensor average, and extend these results to both symmetric and non-symmetric tensor variants, including the large-$p$ and REM limits. They then predict the performance of greedy algorithms (IGP and LAS) and characterize algorithmic hardness through Overlap Gap Property (OGP) bounds and a Franz-Parisi analysis, finding that annealed OGP bounds do not fully capture IGP performance and that FP probes do not neatly explain algorithmic success. The results illuminate how rare, clustered configurations govern the landscape and suggest avenues for extending hardness proofs (e.g., quenched OGP) and exploring other frameworks (Low Degree, SOS) in this analytically tractable, tunable testbed for typical-case hardness in high-dimensional optimization.

Abstract

In this paper we introduce and study the Maximum-Average Subtensor ($p$-MAS) problem, in which one wants to find a subtensor of size $k$ of a given random tensor of size $N$, both of order $p$, with maximum sum of entries. We are motivated by recent work on the matrix case of the problem in which several equilibrium and non-equilibrium properties have been characterized analytically in the asymptotic regime $1 \ll k \ll N$, and a puzzling phenomenon was observed involving the coexistence of a clustered equilibrium phase and an efficient algorithm which produces submatrices in this phase. Here we extend previous results on equilibrium and algorithmic properties for the matrix case to the tensor case. We show that the tensor case has a similar equilibrium phase diagram as the matrix case, and an overall similar phenomenology for the considered algorithms. Additionally, we consider out-of-equilibrium landscape properties using Overlap Gap Properties and Franz-Parisi analysis, and discuss the implications or lack-thereof for average-case algorithmic hardness.

Figures (6)

• Figure 1: Annealed OGP entropies and associated algorithmic hardness OGP upper bounds in the limit $m \to 0$ for the symmetric $p$-MAS problem. (Left) Equilibrium phase transitions $a_{\rm max}(p)$ (maximum subtensor average) and $a_{\rm freezing}(p)$ (freezing) compared with the performance of the greedy LAS and IGP algorithms and with $a^{\rm ann}_{\rm OGP}(p)$, i.e. the largest value of $a$ such that the annealed OGP entropy $s_{\mathrm{ann}}^{(p,r = p)}(l,a) \geq 0$ for $l \in [0,1]$. (Right) Plots of the annealed OGP entropies $s_{\mathrm{ann}}^{(p,r)}(l,a)$ for $p=2,5$ and $a = 1.93, 1.6413$ respectively, as a function of the rescaled overlap $l = q/m$ and for several values of the number of replicas $r = 2, \dots, 6$. As we increase $a$, the $r=p$ entropy is the first to develop a gap, i.e. a region of $l$ where $s_{\mathrm{ann}}^{(p,r)}(l,a) < 0$.
• Figure 2: Behavior of the Franz-Parisi (FP) entropy as a function of the rescaled overlap $c = q/m$. (Left) We start considering a reference configuration in the RS phase, $a_{\rm ref} \leq a_{ \rm freezing} = \sqrt{2}$. We see that in this case the FP entropy is monotonic for all choices of probe submatrix average $a$, and becomes negative at a rescaled overlap $c$ strictly smaller than one, unless $a = a_{\rm ref}$. (Center) On the other hand, when the reference configuration is in the frozen 1-RSB phase, $a_{\rm ref} > a_{ \rm freezing} = \sqrt{2}$, we see that the FP entropy becomes non-monotonic for certain values of the probe submatrix average $a$, and may develop a gap of negative entropy at intermediate values of the rescaled overlap $c$. (Right) Summary of the behavior of the FP entropy in the $(a_{\rm ref}, a)$ plane. The colored dots are color coded with the curves of the central panel. For $a > a_{\rm ref}$ and $a < a_{\rm crit, 1}$ the FP entropy is monotone decreasing, and becomes negative at a value of the overlap $c < 1$. For $a = a_{\rm ref}$ the behavior is the same, but the entropy is exactly zero at overlap $c=1$. For $a_{\rm crit, 1}(a_{\rm ref}) < a < a_{\rm crit, 2}(a_{\rm ref})$, the FP entropy develops a local maximum close to $c=1$, and has no negative entropy gap between $c=0$ and the location of the local maximum. For $a_{\rm crit, 2}(a_{\rm ref}) < a < a_{\rm max} = 2$, the behavior is the same, but with an additional negative entropy gap between $c=0$ and the location of the local maximum. We also highlight with a dot the coordinates of the local maximum of the curve $a_{\rm crit, 1}(a_{\rm ref})$, and with a diamond the coordinates of $a_{\rm crit, 1}(2)$.
• Figure 3: Structure of level and sub-level sets of the submatrix average. (Left) Sub-level FP entropy $\bar{s}_{ \rm FP}$ for reference submatrix average $a_{\rm ref} = 1.75 > a_{\rm freeze}$ and several values of cutoff submatrix average $a_{\rm cut}$ as a function of the rescaled overlap $c = q/m$. We observe a similar phenomenology as for the standard FP entropy, see Figure \ref{['fig:FP']} center. (Center) Complexity/internal entropy curves for planted clusters in submatrix average level sets $a = 1.44, 1.46, 1.5, 1.56, 1.64, 1.7, 1.8, 1.9, 1.98$. We see that in each energy level sets there are frozen clusters (the equilibrium configurations) as well as planted clusters (with non-zero internal entropy) surrounding equilibrium configurations at larger submatrix averages $a' \in (a,2)$. Notice that the horizontal axis scale is quite stretched w.r.t. the vertical axis scale, so that in particular the slope of all the curves is well below $-1$ at all points. (Right) Reproduction of Figure \ref{['fig:FP']} right, in the context of planted clusters. For each value of $a$ considered in the central panel, we plot here the values of $a' \in (a,2)$ from which the curves in the central panel are constructed parametrically (same color means same value of $a$). Notice that level sets $1.44 \lessapprox a \lessapprox 1.48$ present planted clusters only of either very small or very large internal entropy, with a gap in between, due to the non-monotonicity of $a_{\rm crit,1}$ (blue line). For $\sqrt{2} \approx 1.41 < a \lessapprox 1.44$ instead, only clusters with very small internal entropy are present.
• Figure 4: Illustration of the two steps taken by the original $p=2$ IGP algorithm to increase the size of the subtensor from $r$ to $r+1$. For the purpose of the illustration, we draw a submatrix with contiguous rows and columns. In general, this need not be the case, but we can relabel the rows/columns to make it so.
• Figure 5: Illustration of the first two steps needed to increase the size of the subtensor from $r$ to $r+1$ for the non-symmetric IGP algorithm. Here, $r=4$ and $p=3$.