Nonlinear Laplacians: Tunable principal component analysis under directional prior information
Yuxin Ma, Dmitriy Kunisky
TL;DR
This work introduces the sigma-Laplacian, a nonlinear deformation of the observed matrix for PCA-like tasks when directional prior information about the signal is available. By adding a diagonal matrix formed from a bounded nonlinear function of the signal-propagated vector, and compressing to decouple noise, the authors characterize when an outlier eigenvalue emerges (BBP-type transition) under nonlinear perturbations and derive an additive free-convolution description of the bulk spectrum. They prove that, for Gaussian planted submatrix and related models, appropriately chosen nonlinearities yield substantial improvements over direct spectral methods in detection and weak recovery, and they provide practical strategies to design these nonlinearities (hand-tuned, learned via MLP, and black-box optimization). The approach remains simple and spectrally tractable, offering robust performance and transferability across models, with connections to graph neural networks and potential extensions to more general directional priors. Overall, nonlinear Laplacian spectral algorithms offer a principled, flexible, and effective framework for PCA with directional priors, bridging the gap between classic spectral methods and more complex inference algorithms like AMP or BP.
Abstract
We introduce a new family of algorithms for detecting and estimating a rank-one signal from a noisy observation under prior information about that signal's direction, focusing on examples where the signal is known to have entries biased to be positive. Given a matrix observation $\mathbf{Y}$, our algorithms construct a nonlinear Laplacian, another matrix of the form $\mathbf{Y}+\mathrm{diag}(σ(\mathbf{Y1}))$ for a nonlinear $σ:\mathbb{R}\to\mathbb{R}$, and examine the top eigenvalue and eigenvector of this matrix. When $\mathbf{Y}$ is the (suitably normalized) adjacency matrix of a graph, our approach gives a class of algorithms that search for unusually dense subgraphs by computing a spectrum of the graph "deformed" by the degree profile $\mathbf{Y1}$. We study the performance of such algorithms compared to direct spectral algorithms (the case $σ=0$) on models of sparse principal component analysis with biased signals, including the Gaussian planted submatrix problem. For such models, we rigorously characterize the strength of rank-one signal, as a function of $σ$, required for an outlier eigenvalue to appear in the spectrum of a nonlinear Laplacian matrix. While identifying the $σ$ that minimizes the required signal strength in closed form seems intractable, we explore three approaches to design $σ$ numerically: exhaustively searching over simple classes of $σ$, learning $σ$ from datasets of problem instances, and tuning $σ$ using black-box optimization of the critical signal strength. We find both theoretically and empirically that, if $σ$ is chosen appropriately, then nonlinear Laplacian spectral algorithms substantially outperform direct spectral algorithms, while retaining the conceptual simplicity of spectral methods compared to broader classes of computations like approximate message passing or general first order methods.