Modified limited memory BFGS with displacement aggregation and its application to the largest eigenvalue problem

TL;DR

This work introduces AggMBFGS, a modified limited-memory quasi-Newton method that uses displacement aggregation to refine curvature information in nonconvex optimization, achieving per-iteration complexity of $O(\tau d)$ and memory of $O(\tau d)$. It establishes global convergence under a backtracking Modified Armijo line search and proves local superlinear convergence, improving theoretical guarantees over standard M-LBFGS. Empirical tests on CUTEst problems show AggMBFGS reduces iterations and function evaluations compared to M-LBFGS, and it efficiently handles large-scale eigenvalue computations by minimizing $f(x)=\frac{\|x\|^4}{4}-\frac{x^T A x}{2}$ to estimate the largest eigenvalue with lower relative error than baselines. The results suggest AggMBFGS as a practical and scalable tool for large-scale nonconvex optimization and eigenvalue problems, while leaving open questions about worst-case complexity bounds.

Abstract

We present a modified limited memory BFGS method with displacement aggregation (AggMBFGS) for solving nonconvex optimization problems. AggMBFGS refines curvature pair updates by removing linearly dependent variable variations, ensuring that the inverse Hessian approximation retains essential curvature properties. As a result, its per iteration complexity and storage requirement is $\mathcal{O}(τd)$ where $τ\leq d$ represents the memory size and $d$ is the problem dimension. We establish the global convergence of both M-LBFGS and AggMBFGS under a backtracking modified Armijo line search (MALS) and prove the local superlinear convergence of AggMBFGS, demonstrating its theoretical advantages over M-LBFGS with the classical Armijo line search~\cite{Shi2016ALM}. Numerical experiments on CUTEst test problems~\cite{gould2015cutest} confirm that AggMBFGS outperforms M-LBFGS in reducing the number of iterations and function evaluations. Additionally, we apply AggMBFGS to compute the largest eigenvalue of high-dimensional real symmetric positive definite matrices, achieving lower relative errors than M-LBFGS~\cite{Shi2016ALM} while maintaining computational efficiency. These results suggest that AggMBFGS is a promising alternative for large-scale nonconvex optimization and eigenvalue computation.

Theorems & Definitions (15)

• Theorem 2.1
• Remark 1
• Remark 2
• Proposition 3.1
• proof
• Remark 3
• Proposition 3.2
• proof
• Theorem 3.3
• proof