Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Fully Probabilistic Tensor Network for Regularized Volterra System Identification

Afra Kilic, Kim Batselier

TL;DR

The paper tackles nonlinear system identification with Volterra models, whose kernel coefficients grow exponentially with model order. It extends the Bayesian Tensor Network framework to create BTN-V, a fully probabilistic Volterra kernel machine that represents Volterra coefficients via CPD, reducing parameter complexity to $O(DIR)$ and enabling uncertainty quantification without extra cost. Sparsity-inducing hierarchical priors automatically infer the effective tensor rank and learn fading-memory behavior directly from data, improving interpretability and mitigating overfitting. Empirical results on the Cascaded Tanks Benchmark show competitive predictive accuracy, enhanced uncertainty quantification, and substantially reduced computation compared with state-of-the-art methods; the approach also demonstrates automatic rank inference and memory learning. Future work includes extending BTN-V to MIMO systems and exploring alternative priors and tensor-network decompositions.

Abstract

Modeling nonlinear systems with Volterra series is challenging because the number of kernel coefficients grows exponentially with the model order. This work introduces Bayesian Tensor Network Volterra kernel machines (BTN-V), extending the Bayesian Tensor Network framework to Volterra system identification. BTN-V represents Volterra kernels using canonical polyadic decomposition, reducing model complexity from O(I^D) to O(DIR). By treating all tensor components and hyperparameters as random variables, BTN-V provides predictive uncertainty estimation at no additional computational cost. Sparsity-inducing hierarchical priors enable automatic rank determination and the learning of fading-memory behavior directly from data, improving interpretability and preventing overfitting. Empirical results demonstrate competitive accuracy, enhanced uncertainty quantification, and reduced computational cost.

A Fully Probabilistic Tensor Network for Regularized Volterra System Identification

TL;DR

The paper tackles nonlinear system identification with Volterra models, whose kernel coefficients grow exponentially with model order. It extends the Bayesian Tensor Network framework to create BTN-V, a fully probabilistic Volterra kernel machine that represents Volterra coefficients via CPD, reducing parameter complexity to and enabling uncertainty quantification without extra cost. Sparsity-inducing hierarchical priors automatically infer the effective tensor rank and learn fading-memory behavior directly from data, improving interpretability and mitigating overfitting. Empirical results on the Cascaded Tanks Benchmark show competitive predictive accuracy, enhanced uncertainty quantification, and substantially reduced computation compared with state-of-the-art methods; the approach also demonstrates automatic rank inference and memory learning. Future work includes extending BTN-V to MIMO systems and exploring alternative priors and tensor-network decompositions.

Abstract

Modeling nonlinear systems with Volterra series is challenging because the number of kernel coefficients grows exponentially with the model order. This work introduces Bayesian Tensor Network Volterra kernel machines (BTN-V), extending the Bayesian Tensor Network framework to Volterra system identification. BTN-V represents Volterra kernels using canonical polyadic decomposition, reducing model complexity from O(I^D) to O(DIR). By treating all tensor components and hyperparameters as random variables, BTN-V provides predictive uncertainty estimation at no additional computational cost. Sparsity-inducing hierarchical priors enable automatic rank determination and the learning of fading-memory behavior directly from data, improving interpretability and preventing overfitting. Empirical results demonstrate competitive accuracy, enhanced uncertainty quantification, and reduced computational cost.

Paper Structure

This paper contains 16 sections, 22 equations, 3 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Tensor Basics and Notation
  3. Tensor Network SISO Volterra System
  4. Probabilistic Tensor Network SISO Volterra Systems
  5. Probabilistic Model and Priors
  6. Identification Process
  7. Posterior Distribution of Factor Matrices
  8. Posterior distributions of $\bm{\lambda}$ and $\bm{\delta}$
  9. Posterior distribution of noise precision $\tau$
  10. Lower Bound of Model Evidence
  11. Predictive Distribution
  12. Numerical Experiments
  13. Benchmark Description
  14. Regularization Through $\bm{\delta}$ and $\bm{\lambda}$
  15. Predictive Performance
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Probabilistic graphical model of the Volterra TN, where the CPD-decomposed coefficients $\mathbf{w}$ are represented by factor matrices $\{\mathbf{W}^{(d)}\}_{d=1}^D$. Dashed, solid, and unbounded nodes denote observed data $\mathbf{y}$, random variables, and Gamma hyperparameters, respectively..
  • Figure 2: Absolute values of the column weights of the factor matrices without (top) and with (bottom) $\boldsymbol{\delta}$ regularization. The regularization of $\boldsymbol{\delta}$ enforces decay across lags, consistent with the fading memory property of Volterra kernels.
  • Figure 3: Predicted and actual validation outputs for BTN-V and BMVALS. The solid line shows the true observations, the dashed line shows the predictive mean, and the shaded area represents $\pm 3$ standard deviations from the mean.

Theorems & Definitions (2)

  • Definition 1
  • Remark