Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nansde-net: A neural sde framework for generating time series with memory

Hiromu Ozai, Kei Nakagawa

TL;DR

The paper introduces NA-noise, a neural network–kernel ARMA-type noise that is compatible with Itô calculus, enabling memory-aware time-series generation within Neural SDEs. Building on this, NANSDE-Net replaces Brownian noise with NA-noise in neural SDEs to capture both long- and short-memory while preserving the Markov augmentation and allowing efficient training via backpropagation. The authors prove existence and uniqueness of solutions and demonstrate, through synthetic and real-data experiments, that NANSDE-Net can reproduce memory features and yield competitive or superior performance to fSDE-Net, particularly for data with long memory (high Hurst index). They also discuss limitations, such as lack of guaranteed stationary increments, and outline directions for enforcing stationarity and refining kernel designs for explicit long-memory behavior, with code available for replication. Overall, the work offers a principled Itô-compatible alternative to fBm-based approaches for memory modeling in neural SDEs with practical training and inference advantages.

Abstract

Modeling time series with long- or short-memory characteristics is a fundamental challenge in many scientific and engineering domains. While fractional Brownian motion has been widely used as a noise source to capture such memory effects, its incompatibility with Itô calculus limits its applicability in neural stochastic differential equation~(SDE) frameworks. In this paper, we propose a novel class of noise, termed Neural Network-kernel ARMA-type noise~(NA-noise), which is an Itô-process-based alternative capable of capturing both long- and short-memory behaviors. The kernel function defining the noise structure is parameterized via neural networks and decomposed into a product form to preserve the Markov property. Based on this noise process, we develop NANSDE-Net, a generative model that extends Neural SDEs by incorporating NA-noise. We prove the theoretical existence and uniqueness of the solution under mild conditions and derive an efficient backpropagation scheme for training. Empirical results on both synthetic and real-world datasets demonstrate that NANSDE-Net matches or outperforms existing models, including fractional SDE-Net, in reproducing long- and short-memory features of the data, while maintaining computational tractability within the Itô calculus framework.

Nansde-net: A neural sde framework for generating time series with memory

TL;DR

The paper introduces NA-noise, a neural network–kernel ARMA-type noise that is compatible with Itô calculus, enabling memory-aware time-series generation within Neural SDEs. Building on this, NANSDE-Net replaces Brownian noise with NA-noise in neural SDEs to capture both long- and short-memory while preserving the Markov augmentation and allowing efficient training via backpropagation. The authors prove existence and uniqueness of solutions and demonstrate, through synthetic and real-data experiments, that NANSDE-Net can reproduce memory features and yield competitive or superior performance to fSDE-Net, particularly for data with long memory (high Hurst index). They also discuss limitations, such as lack of guaranteed stationary increments, and outline directions for enforcing stationarity and refining kernel designs for explicit long-memory behavior, with code available for replication. Overall, the work offers a principled Itô-compatible alternative to fBm-based approaches for memory modeling in neural SDEs with practical training and inference advantages.

Abstract

Modeling time series with long- or short-memory characteristics is a fundamental challenge in many scientific and engineering domains. While fractional Brownian motion has been widely used as a noise source to capture such memory effects, its incompatibility with Itô calculus limits its applicability in neural stochastic differential equation~(SDE) frameworks. In this paper, we propose a novel class of noise, termed Neural Network-kernel ARMA-type noise~(NA-noise), which is an Itô-process-based alternative capable of capturing both long- and short-memory behaviors. The kernel function defining the noise structure is parameterized via neural networks and decomposed into a product form to preserve the Markov property. Based on this noise process, we develop NANSDE-Net, a generative model that extends Neural SDEs by incorporating NA-noise. We prove the theoretical existence and uniqueness of the solution under mild conditions and derive an efficient backpropagation scheme for training. Empirical results on both synthetic and real-world datasets demonstrate that NANSDE-Net matches or outperforms existing models, including fractional SDE-Net, in reproducing long- and short-memory features of the data, while maintaining computational tractability within the Itô calculus framework.
Paper Structure (17 sections, 3 theorems, 18 equations, 2 figures, 1 table, 1 algorithm)

This paper contains 17 sections, 3 theorems, 18 equations, 2 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Gaussian Processes with Stationary Increments and the Semimartingale Property
  4. ARMA-Type noise with the kernel approximated by a neural network
  5. SDE Driven by NN-kernel ARMA-Type Noise
  6. Generative Modeling of Time Series Using NANSDE-Net
  7. Network Architecture
  8. Algorithm
  9. Experiments
  10. Synthetic data
  11. Real data
  12. Experimental Setup
  13. Performance Metrics
  14. Results
  15. Conclusion
  16. ...and 2 more sections

Key Result

proposition 1

For real parameter $q \in \mathbb R$ and positive parameter $p \in \mathbb R_{>0}$ that satisfy $p > q$, determine $a:(0,\infty) \to \mathbb R$ in (eq:AR-infty-type) as follows: Then, $\{Z(t)\}_{t \in \mathbb R}$ in (eq:AR-infty-type) has the following MA($\infty$)-type representation: Where $c:(0,\infty) \to \mathbb R$ is expressed in the following form:

Figures (2)

  • Figure 1: Figure showing the paths of the original data and generated data for NhemiTemp (Data showing long-memory). Synthetic time series are generated after calibration by RNN (upper center), SDE-Net (upper right), fSDE-Net with $H>1/2$ (lower left) and NANSDE-Net(lower center). The NANSDE-Net model successfully reproduces the characteristics of long-memory in NhemiTemp data.
  • Figure 2: Comparison of the histogram of the log return process for both synthetic and actual S&P 500 index data. The synthetic time series are generated after calibration by RNN, SDE-Net, fSDE-Net, and NANSDE-Net, from left to right, respectively.

Theorems & Definitions (4)

  • proposition 1: inoue2005noise1inoue2005noise2
  • proposition 2: inoue2005noise1inoue2005noise2
  • theorem 1: Informal
  • proof