Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Discrete-Time Stability Analysis of ReLU Feedback Systems via Integral Quadratic Constraints

Sahel Vahedi Noori, Bin Hu, Geir Dullerud, Peter Seiler

TL;DR

This work addresses internal stability of discrete-time Lurye interconnections with a scalar ReLU nonlinearity, motivated by recurrent neural networks. It develops hard integral quadratic constraints (IQCs) tailored to ReLU, deriving a new dynamic IQC that supersets all Zames–Falb multipliers for $[0,1]$ slope-restricted nonlinearities and integrates it into a lifted-LMI stability framework. Compared with prior static QC and FIR-based ZF methods, the proposed ReLU IQCs yield significantly less conservative stability margins, especially as the horizon length increases. The results provide robust, computationally tractable certificates of internal stability for ReLU-based RNN-like feedback systems, with practical implications for safety and reliability in neural-network-inspired control and learning-enabled systems.

Abstract

This paper analyzes internal stability of a discrete-time feedback system with a ReLU nonlinearity. This feedback system is motivated by recurrent neural networks. We first review existing static quadratic constraints (QCs) for slope-restricted nonlinearities. Next, we derive hard integral quadratic constraints (IQCs) for scalar ReLU by using finite impulse filters and structured matrices. These IQCs are combined with a dissipation inequality leading to an LMI condition that certifies internal stability. We show that our new dynamic IQCs for ReLU are a superset of the well-known Zames-Falb IQCs specified for slope-restricted nonlinearities. Numerical results show that the proposed hard IQCs give less conservative stability margins than Zames-Falb multipliers and prior static QC methods, sometimes dramatically so.

Discrete-Time Stability Analysis of ReLU Feedback Systems via Integral Quadratic Constraints

TL;DR

This work addresses internal stability of discrete-time Lurye interconnections with a scalar ReLU nonlinearity, motivated by recurrent neural networks. It develops hard integral quadratic constraints (IQCs) tailored to ReLU, deriving a new dynamic IQC that supersets all Zames–Falb multipliers for slope-restricted nonlinearities and integrates it into a lifted-LMI stability framework. Compared with prior static QC and FIR-based ZF methods, the proposed ReLU IQCs yield significantly less conservative stability margins, especially as the horizon length increases. The results provide robust, computationally tractable certificates of internal stability for ReLU-based RNN-like feedback systems, with practical implications for safety and reliability in neural-network-inspired control and learning-enabled systems.

Abstract

This paper analyzes internal stability of a discrete-time feedback system with a ReLU nonlinearity. This feedback system is motivated by recurrent neural networks. We first review existing static quadratic constraints (QCs) for slope-restricted nonlinearities. Next, we derive hard integral quadratic constraints (IQCs) for scalar ReLU by using finite impulse filters and structured matrices. These IQCs are combined with a dissipation inequality leading to an LMI condition that certifies internal stability. We show that our new dynamic IQCs for ReLU are a superset of the well-known Zames-Falb IQCs specified for slope-restricted nonlinearities. Numerical results show that the proposed hard IQCs give less conservative stability margins than Zames-Falb multipliers and prior static QC methods, sometimes dramatically so.

Paper Structure

This paper contains 14 sections, 6 theorems, 31 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Notation
  3. Problem statement
  4. Preliminary Results
  5. Static Quadratic Constraints
  6. Dynamic Integral Quadratic Constraints
  7. Stability Condition
  8. Main Results
  9. Static Quadratic Constraints
  10. Dynamic Integral Quadratic Constraints
  11. Stability Condition
  12. Numerical examples
  13. Conclusions
  14. Acknowledgments

Key Result

Lemma 1

Let $\Phi:\mathbb{R}^{N} \to \mathbb{R}^{N}$ be a repeated nonlinearity defined element-wise by $\phi: \mathbb{R} \to \mathbb{R}$ where $\phi(0)=0$ and $\phi$ is slope restricted to $[0,1]$. Moreover, let $Q_0 \in \mathbb{R}^{N \times N}$ be any doubly hyperdominant matrix and define Then the following QC holds $\forall v\in\mathbb{R}^{N}$ and $w=\Phi(v)$:

Figures (2)

  • Figure 1: Left: Interconnection $F_U(G,\phi)$ of a nominal discrete-time LTI system $G$ and scalar ReLU $\phi$. Right: Graph of scalar ReLU $\phi$.
  • Figure 2: Lurye System

Theorems & Definitions (13)

  • Lemma 1
  • proof
  • Definition 1
  • Lemma 2
  • proof
  • Theorem 1
  • proof
  • Lemma 3
  • proof
  • Lemma 4
  • ...and 3 more