Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast convolution algorithm for state space models

Gregory Beylkin

TL;DR

The paper develops an unconditionally stable cascade algorithm to apply the transfer operator of a linear time-invariant state-space model in the time domain, addressing instability when the discretized state matrix has eigenvalues near 1. It replaces the inverse rational transfer function $(I_m - z^{-1}\overline{A})^{-1}$ with a finite matrix polynomial $H_N(z)$ of degree $2^{N+1}-1$ via a cascade of factors, enabling stable, low-cost time-domain convolution with only $N+1$ mat-vecs for long sequences. A concrete HiPPO-matrix example demonstrates near-unit eigenvalues and shows that large-degree polynomials can be handled efficiently, with stable behavior and high numerical accuracy; the approach also accommodates structured matrix representations (e.g., PLR, wavelets) to further reduce computational cost. Overall, the method expands the set of feasible structured approximations for long-range dependencies in state-space models by trading a controlled increase in per-step complexity for unconditional stability and flexibility in representation.

Abstract

We present an unconditionally stable algorithm for applying matrix transfer function of a linear time invariant system (LTI) in time domain. The state matrix of an LTI system used for modeling long range dependencies in state space models (SSMs) has eigenvalues close to $1$. The standard recursion defining LTI system becomes unstable if the $m\times m$ state matrix has just one eigenvalue with absolute value even slightly greater than 1. This may occur when approximating a state matrix by a structured matrix to reduce the cost of matrix-vector multiplication from $\mathcal{O}\left(m^{2}\right)$ to $\mathcal{O}\left(m\right)$ or $\mathcal{O}\left(m\log m\right).$ We introduce an unconditionally stable algorithm that uses an approximation of the rational transfer function in the z-domain by a matrix polynomial of degree $2^{N+1}-1$, where $N$ is chosen to achieve any user-selected accuracy. Using a cascade implementation in time domain, applying such transfer function to compute $L$ states requires no more than $2L$ matrix-vector multiplications (whereas the standard recursion requires $L$ matrix-vector multiplications). However, using unconditionally stable algorithm, it is not necessary to assure that an approximate state matrix has all eigenvalues with absolute values strictly less than 1 i.e., within the desired accuracy, the absolute value of some eigenvalues may possibly exceed $1$. Consequently, this algorithm allows one to use a wider variety of structured approximations to reduce the cost of matrix-vector multiplication and we briefly describe several of them to be used for this purpose.

Fast convolution algorithm for state space models

TL;DR

The paper develops an unconditionally stable cascade algorithm to apply the transfer operator of a linear time-invariant state-space model in the time domain, addressing instability when the discretized state matrix has eigenvalues near 1. It replaces the inverse rational transfer function with a finite matrix polynomial of degree via a cascade of factors, enabling stable, low-cost time-domain convolution with only mat-vecs for long sequences. A concrete HiPPO-matrix example demonstrates near-unit eigenvalues and shows that large-degree polynomials can be handled efficiently, with stable behavior and high numerical accuracy; the approach also accommodates structured matrix representations (e.g., PLR, wavelets) to further reduce computational cost. Overall, the method expands the set of feasible structured approximations for long-range dependencies in state-space models by trading a controlled increase in per-step complexity for unconditional stability and flexibility in representation.

Abstract

We present an unconditionally stable algorithm for applying matrix transfer function of a linear time invariant system (LTI) in time domain. The state matrix of an LTI system used for modeling long range dependencies in state space models (SSMs) has eigenvalues close to . The standard recursion defining LTI system becomes unstable if the state matrix has just one eigenvalue with absolute value even slightly greater than 1. This may occur when approximating a state matrix by a structured matrix to reduce the cost of matrix-vector multiplication from to or We introduce an unconditionally stable algorithm that uses an approximation of the rational transfer function in the z-domain by a matrix polynomial of degree , where is chosen to achieve any user-selected accuracy. Using a cascade implementation in time domain, applying such transfer function to compute states requires no more than matrix-vector multiplications (whereas the standard recursion requires matrix-vector multiplications). However, using unconditionally stable algorithm, it is not necessary to assure that an approximate state matrix has all eigenvalues with absolute values strictly less than 1 i.e., within the desired accuracy, the absolute value of some eigenvalues may possibly exceed . Consequently, this algorithm allows one to use a wider variety of structured approximations to reduce the cost of matrix-vector multiplication and we briefly describe several of them to be used for this purpose.

Paper Structure

This paper contains 7 sections, 1 theorem, 22 equations.

Table of Contents

  1. Introduction
  2. Fast cascade algorithm
  3. Convolution via a cascade algorithm
  4. An example of HiPPO matrix
  5. Structured approximations of matrices
  6. Discussion
  7. Acknowledgment

Key Result

Lemma 1

Let $\gamma<1$ be largest singular value of $\overline{A}$ and $N$ a positive integer. Since $\gamma\ge\left|\lambda_{j}\right|$, $j=1,\dots,m$, we estimate and, as a result, we have an approximation for $\left|z\right|=1$, where By taking $N$ large enough, we can achieve any user-selected accuracy.

Theorems & Definitions (1)

  • Lemma 1