Log Neural Controlled Differential Equations: The Lie Brackets Make a Difference

TL;DR

This work introduces Log-NCDEs, a novel method that enhances Neural Controlled Differential Equations by leveraging the Log-ODE approach and iterated Lie brackets to construct a compact, high-fidelity vector field. By proving Lip($oldgamma$) bounds for certain FCNNs and enabling efficient Lie-bracket computations via a Hall basis, the authors achieve substantial improvements in accuracy and efficiency across diverse multivariate time-series benchmarks, including UEA-MTSCA and PPG-DaLiA. The method preserves NCDEs’ robustness to irregular sampling while delivering state-of-the-art performance, validating the importance of Lie-algebraic structure in control-path learning. The results suggest promising directions for extending to higher-depth Log-ODE approximations and adaptive strategies, with broad implications for real-world time-series modelling in fields like healthcare and finance.

Abstract

The vector field of a controlled differential equation (CDE) describes the relationship between a control path and the evolution of a solution path. Neural CDEs (NCDEs) treat time series data as observations from a control path, parameterise a CDE's vector field using a neural network, and use the solution path as a continuously evolving hidden state. As their formulation makes them robust to irregular sampling rates, NCDEs are a powerful approach for modelling real-world data. Building on neural rough differential equations (NRDEs), we introduce Log-NCDEs, a novel, effective, and efficient method for training NCDEs. The core component of Log-NCDEs is the Log-ODE method, a tool from the study of rough paths for approximating a CDE's solution. Log-NCDEs are shown to outperform NCDEs, NRDEs, the linear recurrent unit, S5, and MAMBA on a range of multivariate time series datasets with up to $50{,}000$ observations.

Figures (6)

• Figure 1: A schematic diagram of the Log-ODE method.
• Figure 2: A schematic diagram of the training loop for a Log-NCDE. The coloured circles labelled data observations represent irregular samples from a time series. The purple line labelled system response is a potentially time varying label one wants to predict. The log-signatures of the data observations over each interval $[r_i,r_{i+1}]$ are combined with the iterated Lie brackets of $f_{\theta}$ to produce the vector field $g_{\theta, X}$ from \ref{['eq:Log-NCDE']}. The pink dashed line represents the solution of \ref{['eq:ncde']} and the solid black line represents the approximation obtained by solving \ref{['eq:Log-NCDE']}. A linear map $l_{\psi}$ gives the Log-NCDE's prediction and a loss function $L(\cdot, \cdot)$ is used to update $f_{\theta}$'s parameters.
• Figure 3: A plot of $\beta(v, N)$ against $v$ for $N=1,2$. The output dimension of a NRDE's neural network is $\mathbb{R}^{u\times \beta(v,N)}$, whereas for a Log-NCDE it is $\mathbb{R}^{u \times v}$.
• Figure 4: An example path from the toy dataset, where each colour represents a channel in the path.
• Figure 5: Validation accuracy against number of steps for LRU, S5, S6, MAMBA, NCDE, NRDE, and Log-NCDE on the four different classifications considered for the toy dataset.

Theorems & Definitions (24)

• Definition 2.1
• Definition 2.2
• Definition 2.3
• Definition 2.4
• Definition 2.5
• Definition 2.6
• Definition 2.7
• Theorem 3.1
• proof
• Definition 1.1