Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Introducing Interval Neural Networks for Uncertainty-Aware System Identification

Mehmet Ali Ferah, Tufan Kumbasar

TL;DR

Uncertainty quantification is lacking in DL-based SysID methods that produce point predictions. The authors propose Interval Neural Networks (INNs) that convert pretrained neural parameters into interval LPs and propagate intervals via interval arithmetic to yield Prediction Intervals (PIs) without probabilistic assumptions. They extend LSTM and Neural ODEs to Interval LSTM (ILSTM) and Interval NODE (INODE), and train them with a UQ loss and specialized parameterization tricks, introducing the concept of elasticity to diagnose uncertainty sources. Empirical results on multiple SysID benchmarks show that INODE-2 achieves robust coverage with compact PIs, and elasticity analysis highlights which lagged inputs and weights contribute most to uncertainty.

Abstract

System Identification (SysID) is crucial for modeling and understanding dynamical systems using experimental data. While traditional SysID methods emphasize linear models, their inability to fully capture nonlinear dynamics has driven the adoption of Deep Learning (DL) as a more powerful alternative. However, the lack of uncertainty quantification (UQ) in DL-based models poses challenges for reliability and safety, highlighting the necessity of incorporating UQ. This paper introduces a systematic framework for constructing and learning Interval Neural Networks (INNs) to perform UQ in SysID tasks. INNs are derived by transforming the learnable parameters (LPs) of pre-trained neural networks into interval-valued LPs without relying on probabilistic assumptions. By employing interval arithmetic throughout the network, INNs can generate Prediction Intervals (PIs) that capture target coverage effectively. We extend Long Short-Term Memory (LSTM) and Neural Ordinary Differential Equations (Neural ODEs) into Interval LSTM (ILSTM) and Interval NODE (INODE) architectures, providing the mathematical foundations for their application in SysID. To train INNs, we propose a DL framework that integrates a UQ loss function and parameterization tricks to handle constraints arising from interval LPs. We introduce novel concept "elasticity" for underlying uncertainty causes and validate ILSTM and INODE in SysID experiments, demonstrating their effectiveness.

Introducing Interval Neural Networks for Uncertainty-Aware System Identification

TL;DR

Uncertainty quantification is lacking in DL-based SysID methods that produce point predictions. The authors propose Interval Neural Networks (INNs) that convert pretrained neural parameters into interval LPs and propagate intervals via interval arithmetic to yield Prediction Intervals (PIs) without probabilistic assumptions. They extend LSTM and Neural ODEs to Interval LSTM (ILSTM) and Interval NODE (INODE), and train them with a UQ loss and specialized parameterization tricks, introducing the concept of elasticity to diagnose uncertainty sources. Empirical results on multiple SysID benchmarks show that INODE-2 achieves robust coverage with compact PIs, and elasticity analysis highlights which lagged inputs and weights contribute most to uncertainty.

Abstract

System Identification (SysID) is crucial for modeling and understanding dynamical systems using experimental data. While traditional SysID methods emphasize linear models, their inability to fully capture nonlinear dynamics has driven the adoption of Deep Learning (DL) as a more powerful alternative. However, the lack of uncertainty quantification (UQ) in DL-based models poses challenges for reliability and safety, highlighting the necessity of incorporating UQ. This paper introduces a systematic framework for constructing and learning Interval Neural Networks (INNs) to perform UQ in SysID tasks. INNs are derived by transforming the learnable parameters (LPs) of pre-trained neural networks into interval-valued LPs without relying on probabilistic assumptions. By employing interval arithmetic throughout the network, INNs can generate Prediction Intervals (PIs) that capture target coverage effectively. We extend Long Short-Term Memory (LSTM) and Neural Ordinary Differential Equations (Neural ODEs) into Interval LSTM (ILSTM) and Interval NODE (INODE) architectures, providing the mathematical foundations for their application in SysID. To train INNs, we propose a DL framework that integrates a UQ loss function and parameterization tricks to handle constraints arising from interval LPs. We introduce novel concept "elasticity" for underlying uncertainty causes and validate ILSTM and INODE in SysID experiments, demonstrating their effectiveness.

Paper Structure

This paper contains 15 sections, 36 equations, 4 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries on Interval Arithmetic
  3. Learning NNs for SysID
  4. Feedforward Neural Networks
  5. Long Short Term Memory Network
  6. Neural Ordinary Differential Equations Networks
  7. Interval NNs for UQ: Structure and Inference
  8. Feedforward Interval Neural Network Construction
  9. Inference of Interval LSTM
  10. Inference of Interval NODE
  11. Learning Interval NNs for UQ
  12. Comparative Performance Analysis
  13. Coverage Performance Analysis
  14. Analyzing the Uncertainty Representation
  15. Conclusion and Future Work

Figures (4)

  • Figure 1: Heat Exchanger Dataset: UQ Performance Comparison
  • Figure 2: Hair Dryer Dataset: UQ Performance Comparison
  • Figure 3: MR-Damper Dataset: UQ Performance Comparison
  • Figure 4: Heatmaps of INN LPs from a SysID experiment on the Exchanger Dataset: (a) INDOE-1 and (b) INODE-2

Theorems & Definitions (2)

  • Definition 1
  • Definition 2