Random Linear Systems with Quadratic Constraints: from Random Matrix Theory to replicas and back

Abstract

I present here a pedagogical introduction to the works by Rashel Tublin and Yan V. Fyodorov on random linear systems with quadratic constraints, using tools from Random Matrix Theory and replicas. These notes illustrate and complement the material presented at the Summer School organised within the Puglia Summer Trimester 2023 in Bari (Italy). Consider a system of $M$ linear equations in $N$ unknowns, $\sum_{j=1}^N A_{kj}x_j=b_k$ for $k=1,\ldots,M$, subject to the constraint that the solutions live on the $N$-sphere, $x_1^2+\ldots + x_N^2=N$. Assume that both the coefficients $A_{ij}$ and the parameters $b_i$ be independent Gaussian random variables with zero mean. Using two different approaches -- based on Random Matrix Theory and on a replica calculation -- it is possible to compute whether a large linear system subject to a quadratic constraint is typically solvable or not, as a function of the ratio $α=M/N$ and the variance $σ^2$ of the $b_i$'s. This is done by defining a quadratic loss function $H({\bf x})=\frac{1}{2}\sum_{k=1}^M\left[\sum_{j=1}^NA_{kj} x_j-b_k\right]^2$ and computing the statistics of its minimal value on the sphere, $E_{min}=\min_{||\bf x||^2=N}H({\bf x})$, which is zero if the system is compatible, and larger than zero if it is incompatible. One finds that there exists a compatibility threshold $0<α_c<1$, such that systems with $α>α_c$ are typically incompatible. This means that even weakly under-complete linear systems could become typically incompatible if forced to additionally obey a quadratic constraint.

Figures (5)

• Figure 1: Setting of the orthogonal Procrustes problem. A cloud of red points is first rigidly rotated clockwise via an orthogonal matrix $\Omega$, and some small Gaussian noise is then added to the resulting rotated points, yielding the final blue points. Given the two clouds (red and blue points), one seeks for the "best" orthogonal matrix $\tilde{\Omega}$ that would bring the blue points "as close as possible" to the initial red points.
• Figure 2: Sketch of the plot of the left hand side of Eq. \ref{['spherical2']} as a function of $\lambda$ for $N=5$. The horizontal red line signals the right hand side constant value $N/\sigma^2$, and green points highlight the intersections (solutions of the equation \ref{['spherical2']}). The leftmost intersection (to the left of the smallest eigenvalue $s_1$) corresponds to the minimal cost.
• Figure 3: Red points: histogram of eigenvalues $s_i$ of a Wishart ensemble generated from $500$ "data" matrices $A$ each of size $M\times N$, with $M=500$, $N=200$ (corresponding to $\alpha=2.5$) and filled with i.i.d. Gaussian entries with mean zero and variance $1/N$. Solid black line: the Marčenko-Pastur law \ref{['MPeq']} over the compact support $[s_-,s_+]$, with $s_-=(\sqrt{\alpha}-1)^2\approx 0.338$ and $s_+=(\sqrt{\alpha}+1)^2\approx 6.662$, showing excellent agreement.
• Figure 4: Rate function $\mathcal{L}(x)$ for $\alpha=1.5$ and $\sigma=0.5$, obtained numerically starting from the cubic equation Eq. \ref{['LD_rate_def6']} and using the procedure detailed above.
• Figure 5: Solid blue line: rate function $I(x)$ in Eq. \ref{['eq:rate']}. Solid yellow line: quadratic behaviour $2(x-1/2)^2$ around the minimum (see Eq. \ref{['eq:quadratic']}). This plot -- and the visible deviation between the two curves at the edges -- clearly shows that the Gaussian behaviour around the minimum is inadequate to characterise anomalous events characterised by a very large or small number of Heads in a series of coin tosses.