Efficient Neural Controlled Differential Equations via Attentive Kernel Smoothing

TL;DR

This work tackles the inefficiency of Neural CDEs caused by rough control paths from exact interpolation. It introduces Kernel and Gaussian Process smoothing to decouple solver cost from input noise and presents Multi-View CDE (MV-CDE) and MVC-CDE to recover high-frequency information via attention-guided aggregation of multiple smoothed trajectories. The proposed MVC-CDE (GP) achieves state-of-the-art accuracy on diverse time-series benchmarks while delivering 4.3×–14.5× speedups and reduced NFEs compared to spline-based baselines, with robustness to noise. This approach offers a scalable, continuous-time modeling paradigm by combining principled smoothing priors with learnable multi-view reconstruction and parallel integration.

Abstract

Neural Controlled Differential Equations (Neural CDEs) provide a powerful continuous-time framework for sequence modeling, yet the roughness of the driving control path often restricts their efficiency. Standard splines introduce high-frequency variations that force adaptive solvers to take excessively small steps, driving up the Number of Function Evaluations (NFE). We propose a novel approach to Neural CDE path construction that replaces exact interpolation with Kernel and Gaussian Process (GP) smoothing, enabling explicit control over trajectory regularity. To recover details lost during smoothing, we propose an attention-based Multi-View CDE (MV-CDE) and its convolutional extension (MVC-CDE), which employ learnable queries to inform path reconstruction. This framework allows the model to distribute representational capacity across multiple trajectories, each capturing distinct temporal patterns. Empirical results demonstrate that our method, MVC-CDE with GP, achieves state-of-the-art accuracy while significantly reducing NFEs and total inference time compared to spline-based baselines.

Figures (10)

• Figure 1: Standard Neural CDE pipeline for different types of path construction.
• Figure 2: Proposed MV-CDE and MVC-CDE architectures.
• Figure 3: Pareto Efficiency Plot. Error Rate (log scale) versus Total Training Time (log scale).
• Figure 4: Noise Robustness Analysis. The plots compare the stability of different path-construction methods under varying intensities of additive white Gaussian noise. Top row: Relative NFE normalized to the noise-free number. Bottom row: Classification error rate.
• Figure 5: Learned Attention Weights across Datasets. The heatmaps visualize the example of learned attention weights for one sample example assigned to different smoothing heads over time.

Theorems & Definitions (10)

• Theorem 3.1: NFE dependence on Control Path (informal)
• Theorem 3.2: Derivative Bounds for Smoothing Kernels
• Corollary 3.3: Smoothing NFE Scaling
• Theorem 3.4: Parallel Integration Bottleneck
• Theorem : \ref{['thm:step_size']}
• Theorem : \ref{['thm:smoothing_bounds']}
• proof
• Corollary : \ref{['nfe']}
• Theorem : \ref{['thm:parallel_bottleneck']}
• proof