Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

State-space systems as dynamic generative models

Juan-Pablo Ortega, Florian Rossmannek

TL;DR

The paper develops a probabilistic framework for state-space systems that generalizes the deterministic echo state property to stochastic inputs by formulating a stochastic echo state property via fixed points of the lifted map $\mathcal{F}_*$ on joint input-output laws. It introduces stochastic contractivity based on input laws and a boundedness notion, and proves a Stochastic Echo State Theorem: for $C>0$ and $\kappa \in (0,2^{1-p}|\underline{w}|^{-1})$, every input law in $\mathcal{M}_p^{C,\kappa}(\underline{\mathcal{U}})$ yields a unique $V$-causal, $\underline{w}$-weighted fixed point $\mu^{\Xi}$ that depends continuously on the input in the Wasserstein metric $W_p$. The framework supports concrete instances such as GARCH-type state-space models, time-varying VAR/state-affine systems, and echo state networks, showing that stochastic contracts can yield a probabilistic dependence structure even without a deterministic functional relation. The results imply that the state-space system can act as a dynamic generative model for covariate-output dependencies and provide a pathway to learning stochastic dynamic dependencies with reduced contraction requirements compared to the deterministic setting. The paper also develops an abstract fixed-point theory on Wasserstein spaces and discusses extensions, including fixed points for hidden inputs and considerations of universality and causality in ESN contexts.

Abstract

A probabilistic framework to study the dependence structure induced by deterministic discrete-time state-space systems between input and output processes is introduced. General sufficient conditions are formulated under which output processes exist and are unique once an input process has been fixed, a property that in the deterministic state-space literature is known as the echo state property. When those conditions are satisfied, the given state-space system becomes a generative model for probabilistic dependences between two sequence spaces. Moreover, those conditions guarantee that the output depends continuously on the input when using the Wasserstein metric. The output processes whose existence is proved are shown to be causal in a specific sense and to generalize those studied in purely deterministic situations. The results in this paper constitute a significant stochastic generalization of sufficient conditions for the deterministic echo state property to hold, in the sense that the stochastic echo state property can be satisfied under contractivity conditions that are strictly weaker than those in deterministic situations. This means that state-space systems can induce a purely probabilistic dependence structure between input and output sequence spaces even when there is no functional relation between those two spaces.

State-space systems as dynamic generative models

TL;DR

The paper develops a probabilistic framework for state-space systems that generalizes the deterministic echo state property to stochastic inputs by formulating a stochastic echo state property via fixed points of the lifted map on joint input-output laws. It introduces stochastic contractivity based on input laws and a boundedness notion, and proves a Stochastic Echo State Theorem: for and , every input law in yields a unique -causal, -weighted fixed point that depends continuously on the input in the Wasserstein metric . The framework supports concrete instances such as GARCH-type state-space models, time-varying VAR/state-affine systems, and echo state networks, showing that stochastic contracts can yield a probabilistic dependence structure even without a deterministic functional relation. The results imply that the state-space system can act as a dynamic generative model for covariate-output dependencies and provide a pathway to learning stochastic dynamic dependencies with reduced contraction requirements compared to the deterministic setting. The paper also develops an abstract fixed-point theory on Wasserstein spaces and discusses extensions, including fixed points for hidden inputs and considerations of universality and causality in ESN contexts.

Abstract

A probabilistic framework to study the dependence structure induced by deterministic discrete-time state-space systems between input and output processes is introduced. General sufficient conditions are formulated under which output processes exist and are unique once an input process has been fixed, a property that in the deterministic state-space literature is known as the echo state property. When those conditions are satisfied, the given state-space system becomes a generative model for probabilistic dependences between two sequence spaces. Moreover, those conditions guarantee that the output depends continuously on the input when using the Wasserstein metric. The output processes whose existence is proved are shown to be causal in a specific sense and to generalize those studied in purely deterministic situations. The results in this paper constitute a significant stochastic generalization of sufficient conditions for the deterministic echo state property to hold, in the sense that the stochastic echo state property can be satisfied under contractivity conditions that are strictly weaker than those in deterministic situations. This means that state-space systems can induce a purely probabilistic dependence structure between input and output sequence spaces even when there is no functional relation between those two spaces.
Paper Structure (18 sections, 14 theorems, 56 equations)

This paper contains 18 sections, 14 theorems, 56 equations.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Inputs generation
  4. Sequence spaces
  5. Wasserstein spaces
  6. The stochastic echo state property
  7. Contractive and bounded input measures
  8. The main result
  9. Proof of the stochastic echo state theorem
  10. Relatively unique fixed points in an abstract framework
  11. Proof of the main result
  12. Conclusion
  13. Fixed points for hidden inputs
  14. Proofs of remarks
  15. On echo state networks
  16. ...and 3 more sections

Key Result

Theorem 11

Let $C \in (0,\infty)$ and $\kappa \in (0,2^{1-p} \left|\underline{w}\right|^{-1})$. For any $\Xi \in \mathcal{M}_p^{C,\kappa}(\underline{\mathcal{U}})$, the map $\mathcal{F}_*$ has a unique fixed point $\mu^{\Xi}$ in $P^{\Xi} \cap P^{V\text{-}\mathrm{causal}} \cap P_p^{\underline{w}}(\underline{\ma

Theorems & Definitions (42)

  • Remark 2
  • Definition 3
  • Definition 4: Stochastic contractivity and boundedness
  • Remark 5
  • Example 6: GARCH process
  • Example 7: Time-varying VAR model/state-affine system
  • Example 8: Echo state networks (ESNs)
  • Example 9: Stochastic difference equation
  • Definition 10
  • Theorem 11: Stochastic echo state theorem
  • ...and 32 more