Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Batch and Recursive Least Squares for Matrix Parameter Estimation

Brian Lai, Dennis S. Bernstein

TL;DR

The paper addresses matrix-structured parameter identification in linear measurement models and shows that vec-permutation via the Kronecker product introduces unnecessary computational and storage burdens. It develops matrix-based batch and recursive least squares (BLS/RLS) formulations that minimize the same cost as the vec-permutation approach under mild independence assumptions, providing substantial reductions in complexity (up to ${\cal O}(m^3)$ time and ${\cal O}(m^2)$ space) while preserving convergence guarantees under persistent excitation. A hierarchy of methods—column-by-column LS with independent column weighting and a matrix update LS with identical weighting—balances performance and efficiency, with rigorous update equations and convergence results. The approaches are demonstrated in online identification for indirect adaptive model predictive control (PCAC), showing dramatic speedups in MIMO online learning (e.g., near-100% per-step time reductions in a truss example), underscoring practical impact for real-time adaptive control. The work also discusses tradeoffs when measurement-noise columns are highly correlated and points to future enhancements such as integrating forgetting factors.

Abstract

Traditionally, batch least squares (BLS) and recursive least squares (RLS) are used for identification of a vector of parameters that form a linear model. In some situations, however, it is of interest to identify parameters in a matrix structure. In this case, a common approach is to transform the problem into standard vector form using the vectorization (vec) operator and the Kronecker product, known as vec-permutation. However, the use of the Kronecker product introduces extraneous zero terms in the regressor, resulting in unnecessary additional computational and space requirements. This work derives matrix BLS and RLS formulations which, under mild assumptions, minimize the same cost as the vec-permutation approach. This new approach requires less computational complexity and space complexity than vec-permutation in both BLS and RLS identification. It is also shown that persistent excitation guarantees convergence to the true matrix parameters. This method can used to improve computation time in the online identification of multiple-input, multiple-output systems for indirect adaptive model predictive control.

Efficient Batch and Recursive Least Squares for Matrix Parameter Estimation

TL;DR

The paper addresses matrix-structured parameter identification in linear measurement models and shows that vec-permutation via the Kronecker product introduces unnecessary computational and storage burdens. It develops matrix-based batch and recursive least squares (BLS/RLS) formulations that minimize the same cost as the vec-permutation approach under mild independence assumptions, providing substantial reductions in complexity (up to time and space) while preserving convergence guarantees under persistent excitation. A hierarchy of methods—column-by-column LS with independent column weighting and a matrix update LS with identical weighting—balances performance and efficiency, with rigorous update equations and convergence results. The approaches are demonstrated in online identification for indirect adaptive model predictive control (PCAC), showing dramatic speedups in MIMO online learning (e.g., near-100% per-step time reductions in a truss example), underscoring practical impact for real-time adaptive control. The work also discusses tradeoffs when measurement-noise columns are highly correlated and points to future enhancements such as integrating forgetting factors.

Abstract

Traditionally, batch least squares (BLS) and recursive least squares (RLS) are used for identification of a vector of parameters that form a linear model. In some situations, however, it is of interest to identify parameters in a matrix structure. In this case, a common approach is to transform the problem into standard vector form using the vectorization (vec) operator and the Kronecker product, known as vec-permutation. However, the use of the Kronecker product introduces extraneous zero terms in the regressor, resulting in unnecessary additional computational and space requirements. This work derives matrix BLS and RLS formulations which, under mild assumptions, minimize the same cost as the vec-permutation approach. This new approach requires less computational complexity and space complexity than vec-permutation in both BLS and RLS identification. It is also shown that persistent excitation guarantees convergence to the true matrix parameters. This method can used to improve computation time in the online identification of multiple-input, multiple-output systems for indirect adaptive model predictive control.
Paper Structure (8 sections, 7 theorems, 44 equations, 3 figures, 3 tables)

This paper contains 8 sections, 7 theorems, 44 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Vec-permutation Least Squares
  3. Column-by-Column Least Squares
  4. Matrix Update Least Squares
  5. Convergence of Matrix Update RLS
  6. Computational Complexity and Performance Tradeoff
  7. Application to Online Identification for Indirect Adaptive Model Predictive Control
  8. Conclusions

Key Result

Proposition 1

For all $k \ge 0$, let $\phi_k \in {\mathbb R}^{p \times n}$, let $y_k \in {\mathbb R}^{p \times m}$, and let $\bar{\Gamma}_k \in {\mathbb R}^{mp \times mp}$ be positive definite. Furthermore, let $\bar{\theta}_0 \in {\mathbb R}^{n \times m}$ and let $\bar{R} \in {\mathbb R}^{mn \times mn}$ be posit where and where, for all $k \ge 0$, $\bar{y}_k \in {\mathbb R}^{mp}$ and $\bar{\phi}_k \in {\mathb

Figures (3)

  • Figure 1: Consider the measurement process \ref{['eqn: yk = phik theta']} with $p = 10$, $n = 50$, and $1 \le m \le 20$. Batch least squares (top) shows computation time with $N = 100$ data points, averaged over 10 trials. Recursive least squares (bottom) shows computation time per step, averaged over 100 trials. Error bars show the 95 confidence intervals.
  • Figure 2: Example \ref{['example: correlated noise']}: Parameter estimation error $\Vert e_k \Vert_2$ over $200$ steps for matrix update, column-by-column, and vec-permutation recursive least squares with 95 confidence intervals over 10 trials highlighted.
  • Figure 3: Flexible truss structure from mohseni2022predictive with nodes labeled.

Theorems & Definitions (13)

  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Proposition 3
  • proof
  • Definition 1
  • Theorem 1
  • Proposition 4
  • proof
  • ...and 3 more