Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Signature Reconstruction from Randomized Signatures

Mie Glückstad, Nicola Muca Cirone, Josef Teichmann

TL;DR

This work investigates how signature features of a bounded-variation path $X$ can be recovered from controlled differential equations driven by randomized neural-like vector fields, with a focus on depth-two constructions. By representing iterated vector-field operators as tree-like structures and employing Taylor-expansion techniques, the authors show that, under a linear-independence condition on these tree-like fields up to order $L$, signature components up to order $L$ can be reconstructed from the family of CDE solutions, and that depth-two randomizations can render these components independent up to higher orders when $N\ge L-1$. The results extend to Lie groups, yielding a general framework that connects randomized signatures, rough-path theory, and algebraic properties of vector-field Lie polynomials. The findings quantify how increasing hidden dimension $N$ yields an exponential growth in the number of reconstructible signature features, with implications for understanding the expressive power of randomized neural-like CDEs in capturing path information. These insights bridge Lie-algebraic structure and machine-learning inspired models, offering pathways for robust feature extraction from sequential data.

Abstract

Controlled ordinary differential equations driven by continuous bounded variation curves can be considered a continuous time analogue of recurrent neural networks for the construction of expressive features of the input curves. We ask up to which extent well known signature features of such curves can be reconstructed from controlled ordinary differential equations with (untrained) random vector fields. The answer turns out to be algebraically involved, but essentially the number of signature features, which can be reconstructed from the non-linear flow of the controlled ordinary differential equation, is exponential in its hidden dimension, when the vector fields are chosen to be neural with depth two. Moreover, we characterize a general linear independence condition on arbitrary vector fields, under which the signature features up to some fixed order can always be reconstructed. Algebraically speaking this complements in a quantitative manner several well known results from the theory of Lie algebras of vector fields and puts them in a context of machine learning.

Signature Reconstruction from Randomized Signatures

TL;DR

This work investigates how signature features of a bounded-variation path can be recovered from controlled differential equations driven by randomized neural-like vector fields, with a focus on depth-two constructions. By representing iterated vector-field operators as tree-like structures and employing Taylor-expansion techniques, the authors show that, under a linear-independence condition on these tree-like fields up to order , signature components up to order can be reconstructed from the family of CDE solutions, and that depth-two randomizations can render these components independent up to higher orders when . The results extend to Lie groups, yielding a general framework that connects randomized signatures, rough-path theory, and algebraic properties of vector-field Lie polynomials. The findings quantify how increasing hidden dimension yields an exponential growth in the number of reconstructible signature features, with implications for understanding the expressive power of randomized neural-like CDEs in capturing path information. These insights bridge Lie-algebraic structure and machine-learning inspired models, offering pathways for robust feature extraction from sequential data.

Abstract

Controlled ordinary differential equations driven by continuous bounded variation curves can be considered a continuous time analogue of recurrent neural networks for the construction of expressive features of the input curves. We ask up to which extent well known signature features of such curves can be reconstructed from controlled ordinary differential equations with (untrained) random vector fields. The answer turns out to be algebraically involved, but essentially the number of signature features, which can be reconstructed from the non-linear flow of the controlled ordinary differential equation, is exponential in its hidden dimension, when the vector fields are chosen to be neural with depth two. Moreover, we characterize a general linear independence condition on arbitrary vector fields, under which the signature features up to some fixed order can always be reconstructed. Algebraically speaking this complements in a quantitative manner several well known results from the theory of Lie algebras of vector fields and puts them in a context of machine learning.

Paper Structure

This paper contains 25 sections, 19 theorems, 90 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Setting
  3. Signatures and rough paths
  4. Taylor expansions
  5. The $\mathbb{R}^N$-case
  6. The Lie group case
  7. Randomized signatures with depth
  8. Linear independence of iterated vector fields on $\mathbb{R}^N$
  9. Tree-like vector fields
  10. Linear independence of iterated vector fields
  11. Example: Nested exponentials
  12. Neural depth-two vector fields
  13. Nested exponentials
  14. Signature reconstruction
  15. Signature reconstruction on $\mathbb{R}^N$
  16. ...and 10 more sections

Key Result

Theorem 2.2

Let $Y: [0,T] \rightarrow \mathbb{R}^N$ denote the unique solution to the differential equation eq:RNDE with initial value $y \in \mathbb{R}^N$, and let $g \in C^{\infty}(\mathbb{R}^N)$. Then:

Figures (7)

  • Figure 1: Example of a planar rooted tree in $\mathbb{T}$ with vertices $\{1, \dots, 7\}$ labeled by the word $w=(w_1, \dots, w_7) \in \mathscr{W}_7$.
  • Figure 2: Left to right, the spaces $\mathbb{T}_{w_1}$, $\mathbb{T}_{w_1w_2}$, $\mathbb{T}_{w_1w_2w_3}$, and $\mathbb{T}_{w_1w_2w_3w_4}$.
  • Figure 3: The planar rooted tree from Figure \ref{['fig:planar_tree_ex']} with assigned edge-directions $j_1, \dots, j_6 \in \{1, \dots, N\}$.
  • Figure 4: Left to right, the spaces $\mathbb{T}_{w_1}^0$, $\mathbb{T}_{(w_1,w_2)}^0$, $\mathbb{T}_{(w_1, w_2, w_3)}^0$.
  • Figure 5: Form of the trees $\tau^0 \in \Pi$.
  • ...and 2 more figures

Theorems & Definitions (45)

  • Definition 2.1: Signature
  • Theorem 2.2
  • Remark 2.3
  • Theorem 2.4
  • Remark 2.5
  • Definition 3.1: Letter-labeled recursive trees
  • Definition 3.2: Tree-like vector fields
  • Definition 3.3: Tree-like operators
  • Lemma 3.4
  • proof
  • ...and 35 more