Krylov iterative methods for linear least squares problems with linear equality constraints

TL;DR

The paper tackles linear equality constrained least squares (LSE) by reframing it as two operator-based LS problems and proving a decomposed-form for the minimum-norm solution. It introduces two Krylov-based decomposed solvers, KIDS-I and KIDS-II, that avoid matrix factorizations through nested inner-outer iterations using gLSQR and NSR-GKB techniques. The approach yields efficient, scalable solutions, with four numerical experiments showing linear convergence and competitive accuracy against classical NS and DE baselines. This work advances large-scale LSE computation by enabling factorization-free, structure-exploiting iterative methods with clear decomposed-form interpretations. Overall, the methods offer practical tools for solving LSE problems in data-fitting, control, and optimization contexts where traditional approaches are impractical due to scale.

Abstract

We consider the linear least squares problem with linear equality constraints (LSE problem) formulated as $\min_{x\in\mathbb{R}^{n}}\|Ax-b\|_2 \ \mathrm{s.t.} \ Cx = d$. Although there are some classical methods available to solve this problem, most of them rely on matrix factorizations or require the null space of $C$, which limits their applicability to large-scale problems. To address this challenge, we present a novel analysis of the LSE problem from the perspective of operator-type least squares (LS) problems, where the linear operators are induced by $\{A,C\}$. We show that the solution of the LSE problem can be decomposed into two components, each corresponding to the solution of an operator-form LS problem. Building on this decomposed-form solution, we propose two Krylov subspace based iterative methods to approximate each component, thereby providing an approximate solution of the LSE problem. Several numerical examples are constructed to test the proposed iterative algorithm for solving the LSE problems, which demonstrate the effectiveness of the algorithms.

Figures (4)

• Figure 5.1: The convergence history of KIDS-I and KIDS-II with respect to the true solution, where all the inner iterations are computed accurately. (a) {$D_1$, lp_bnl2}; (b){$D_2$, r05}; (c) { cage9-I, cage9-II}; (d) { pf2177-I, pf2177-II}.
• Figure 5.2: Curves for the true and computed solutions obtained by KIDS-I at the final iteration. (a) {$D_1$, lp_bnl2}; (b){$D_2$, r05}; (c) { cage9-I, cage9-II}; (d) { pf2177-I, pf2177-II}.
• Figure 5.3: The convergence history of KIDS-I and KIDS-II with respect to the true solution, where the inner iterations are approximated by solving \ref{['ls1_gkb']} and \ref{['ls2_gkb']} by LSQR with stopping tolerance $\tau$. (a) {$D_1$, lp_bnl2}, $\tau=10^{-10}$; (b){$D_1$, lp_bnl2}, $\tau=10^{-8}$; (c) { cage9-I, cage9-II}, $\tau=10^{-10}$; (d) { cage9-I, cage9-II}, $\tau=10^{-8}$.
• Figure 5.4: The convergence history of KIDS-II with respect to the true solution, where $\tilde{x}_{1}^{\dag}=\mathop{\mathrm{argmin}}_{x}\|Cx-d\|_2$ is computed by LSQR with stopping tolerance $\tau_1$, and the inner iteration is approximated by solving \ref{['ls2_gkb']} by LSQR with stopping tolerance $\tau_2$. The test example is {$D_1$, lp_bnl2}.

Theorems & Definitions (19)

• Theorem 3.1
• Theorem 3.2
• proof
• Theorem 3.3
• Lemma 3.1
• proof
• proof : Proof of \ref{['lem:NSLS']}
• Proposition 3.1
• proof
• Corollary 3.1