Revisiting a Fast Newton Solver for a 2-D Spectral Estimation Problem: Computations with the Full Hessian

TL;DR

The paper tackles 2-D spectral estimation through a convex dual formulation and reveals that the full Hessian of the dual objective $J_\Psi$ possesses a Toeplitz-block Toeplitz structure when dual variables are ordered appropriately, enabling a true Newton method. By constructing nested recursions and exploiting Hermitian persymmetry, the authors develop a fast inversion technique and a corresponding solver for TBT linear systems, achieving favorable computational complexity $O(p^5)$ relative to naive $O(p^6)$. Numerical experiments show that Newton's method with the full Hessian converges much faster than the previous quarter-Hessian quasi-Newton, while the fast Hessian inversion remains efficient for large problem sizes (e.g., $p=2n+1$ with $n$ up to 40). Overall, the method offers a substantially faster and scalable approach to solving the dual problem in 2-D spectral estimation. $J_\Psi$, $\mathbf q$, and $Q(\boldsymbol{\theta})$ are central to the dual formulation and its Hessian structure, which underpins the speed gains.

Abstract

Spectral estimation plays a fundamental role in frequency-domain identification and related signal processing problems. This paper revisits a 2-D spectral estimation problem formulated in terms of convex optimization. More precisely, we work with the dual optimization problem and show that the full Hessian of the dual function admits a Toeplitz-block Toeplitz structure which is consistent with our finding in a previous work. This particular structure of the Hessian enables a fast inversion algorithm in the solution of the dual optimization problem via Newton's method whose superior speed of convergence is illustrated via simulations.

Figures (2)

• Figure 1: Algorithm \ref{['alg:Inversion']} vs. the direction method for the inversion of positive definite TBT matrices: average running times for $n$ ranging from $21$ to $40$.
• Figure 2: Full Hessian vs. the quarter Hessian in Newton's method for the solution of the dual problem \ref{['opt_dual']}: Euclidean distance between the $k$-th iterate $\mathbf q^{k}$ and the optimal point $\mathbf q^*$.