Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Time-changed normalizing flows for accurate SDE modeling

Naoufal El Bekri, Lucas Drumetz, Franck Vermet

TL;DR

This work tackles the challenge of modeling stochastic dynamics with neural normalizing flows by addressing a limitation of prior dynamic NF methods that rely on a Brownian base and struggle to represent basic SDEs like the Ornstein-Uhlenbeck process. It introduces Time-changed Normalizing Flow (TCNF), where $X_t = f_\theta\big(W_{\phi(t)},\phi(t)\big)$ and the time-change $\phi$ is a monotone function implemented with a convex neural network. Training maximizes the log-likelihood using the change-of-variables formula with conditional Gaussian increments of variance $\phi(t_i)-\phi(t_{i-1})$, and experiments show improved accuracy over CTFP on both toy OU-type processes and real-world data (Crypto, ECL). The approach preserves the advantages of CNFs—exact density estimation and efficient sampling—while expanding the class of Gaussian processes that can be modeled, with potential extensions to higher dimensions and refined time-change calibration.

Abstract

The generative paradigm has become increasingly important in machine learning and deep learning models. Among popular generative models are normalizing flows, which enable exact likelihood estimation by transforming a base distribution through diffeomorphic transformations. Extending the normalizing flow framework to handle time-indexed flows gave dynamic normalizing flows, a powerful tool to model time series, stochastic processes, and neural stochastic differential equations (SDEs). In this work, we propose a novel variant of dynamic normalizing flows, a Time Changed Normalizing Flow (TCNF), based on time deformation of a Brownian motion which constitutes a versatile and extensive family of Gaussian processes. This approach enables us to effectively model some SDEs, that cannot be modeled otherwise, including standard ones such as the well-known Ornstein-Uhlenbeck process, and generalizes prior methodologies, leading to improved results and better inference and prediction capability.

Time-changed normalizing flows for accurate SDE modeling

TL;DR

This work tackles the challenge of modeling stochastic dynamics with neural normalizing flows by addressing a limitation of prior dynamic NF methods that rely on a Brownian base and struggle to represent basic SDEs like the Ornstein-Uhlenbeck process. It introduces Time-changed Normalizing Flow (TCNF), where and the time-change is a monotone function implemented with a convex neural network. Training maximizes the log-likelihood using the change-of-variables formula with conditional Gaussian increments of variance , and experiments show improved accuracy over CTFP on both toy OU-type processes and real-world data (Crypto, ECL). The approach preserves the advantages of CNFs—exact density estimation and efficient sampling—while expanding the class of Gaussian processes that can be modeled, with potential extensions to higher dimensions and refined time-change calibration.

Abstract

The generative paradigm has become increasingly important in machine learning and deep learning models. Among popular generative models are normalizing flows, which enable exact likelihood estimation by transforming a base distribution through diffeomorphic transformations. Extending the normalizing flow framework to handle time-indexed flows gave dynamic normalizing flows, a powerful tool to model time series, stochastic processes, and neural stochastic differential equations (SDEs). In this work, we propose a novel variant of dynamic normalizing flows, a Time Changed Normalizing Flow (TCNF), based on time deformation of a Brownian motion which constitutes a versatile and extensive family of Gaussian processes. This approach enables us to effectively model some SDEs, that cannot be modeled otherwise, including standard ones such as the well-known Ornstein-Uhlenbeck process, and generalizes prior methodologies, leading to improved results and better inference and prediction capability.
Paper Structure (12 sections, 11 equations, 2 figures, 2 tables)

This paper contains 12 sections, 11 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Neural Stochastic Differential Equations
  4. Normalizing flows
  5. Dynamic Normalizing flow with time-change
  6. Time-changed Normalizing Flow
  7. Time-change function
  8. Training algorithm
  9. Experiments
  10. Toy datasets
  11. Real-world datasets
  12. Conclusion

Figures (2)

  • Figure 1: A time-change $\phi$ is applied to the Wiener process (red) to create a new Gaussian process, which is then mapped through a bijection $f_\theta$ to the observed process (blue).
  • Figure 2: Comparison between TCNF and CTFP on toy-SDE1 (top) and toy-SDE2 (bottom). In each figure we depict density MAE between the flow models and the ground truth (GT), along the mean (continuous line) and the IQR (dashed line).