Statistical-Computational Trade-offs in Learning Multi-Index Models via Harmonic Analysis
Hugo Latourelle-Vigeant, Theodor Misiakiewicz
TL;DR
This work develops a harmonic-analytic framework for learning multi-index models (MIMs) under ${\mathcal{O}}_d$-equivariance with spherically symmetric inputs. By decomposing the problem into spherical harmonic subspaces and exploiting the intertwining with traceless symmetric tensors, the authors establish sharp Statistical Query (SQ) and Low-Degree Polynomial (LDP) lower bounds (leap complexity) and propose iterative harmonic tensor unfolding algorithms that nearly attain these bounds. The proposed methods allow flexible trade-offs between sample complexity and runtime by selecting a sequence of harmonic degrees, and they specialize to Gaussian and directional MIMs, yielding tight or near-tight bounds in these canonical settings. The results provide a principled, symmetry-driven roadmap for constructing statistically and computationally optimal equivariant learning algorithms, with potential extensions to broader group actions and equivariant learning tasks.
Abstract
We study the problem of learning multi-index models (MIMs), where the label depends on the input $\boldsymbol{x} \in \mathbb{R}^d$ only through an unknown $\mathsf{s}$-dimensional projection $\boldsymbol{W}_*^\mathsf{T} \boldsymbol{x} \in \mathbb{R}^\mathsf{s}$. Exploiting the equivariance of this problem under the orthogonal group $\mathcal{O}_d$, we obtain a sharp harmonic-analytic characterization of the learning complexity for MIMs with spherically symmetric inputs -- which refines and generalizes previous Gaussian-specific analyses. Specifically, we derive statistical and computational complexity lower bounds within the Statistical Query (SQ) and Low-Degree Polynomial (LDP) frameworks. These bounds decompose naturally across spherical harmonic subspaces. Guided by this decomposition, we construct a family of spectral algorithms based on harmonic tensor unfolding that sequentially recover the latent directions and (nearly) achieve these SQ and LDP lower bounds. Depending on the choice of harmonic degree sequence, these estimators can realize a broad range of trade-offs between sample and runtime complexity. From a technical standpoint, our results build on the semisimple decomposition of the $\mathcal{O}_d$-action on $L^2 (\mathbb{S}^{d-1})$ and the intertwining isomorphism between spherical harmonics and traceless symmetric tensors.