Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AdaNODEs: Test Time Adaptation for Time Series Forecasting Using Neural ODEs

Ting Dang, Soumyajit Chatterjee, Hong Jia, Yu Wu, Flora Salim, Fahim Kawsar

TL;DR

The paper addresses forecasting under unseen distribution shifts without access to source data by proposing AdaNODEs, a test-time adaptation framework for time series that leverages latent Neural ODEs. It freezes base parameters during inference and introduces two learnable test-time modifiers, $\alpha$ and $\gamma$, to adapt latent dynamics; it uses a probabilistic loss based on negative log-likelihood and KL divergence within an amortized variational inference setting. Empirical results on one-dimensional signals and Rotating MNIST demonstrate consistent improvements over state-of-the-art baselines, particularly under severe shifts. This work establishes a practical, memory-efficient approach to TTA for regression tasks in time-series and highlights the potential of NODE-based architectures for robust adaptation.

Abstract

Test time adaptation (TTA) has emerged as a promising solution to adapt pre-trained models to new, unseen data distributions using unlabeled target domain data. However, most TTA methods are designed for independent data, often overlooking the time series data and rarely addressing forecasting tasks. This paper presents AdaNODEs, an innovative source-free TTA method tailored explicitly for time series forecasting. By leveraging Neural Ordinary Differential Equations (NODEs), we propose a novel adaptation framework that accommodates the unique characteristics of distribution shifts in time series data. Moreover, we innovatively propose a new loss function to tackle TTA for forecasting tasks. AdaNODEs only requires updating limited model parameters, showing effectiveness in capturing temporal dependencies while avoiding significant memory usage. Extensive experiments with one- and high-dimensional data demonstrate that AdaNODEs offer relative improvements of 5.88\% and 28.4\% over the SOTA baselines, especially demonstrating robustness across higher severity distribution shifts.

AdaNODEs: Test Time Adaptation for Time Series Forecasting Using Neural ODEs

TL;DR

The paper addresses forecasting under unseen distribution shifts without access to source data by proposing AdaNODEs, a test-time adaptation framework for time series that leverages latent Neural ODEs. It freezes base parameters during inference and introduces two learnable test-time modifiers, and , to adapt latent dynamics; it uses a probabilistic loss based on negative log-likelihood and KL divergence within an amortized variational inference setting. Empirical results on one-dimensional signals and Rotating MNIST demonstrate consistent improvements over state-of-the-art baselines, particularly under severe shifts. This work establishes a practical, memory-efficient approach to TTA for regression tasks in time-series and highlights the potential of NODE-based architectures for robust adaptation.

Abstract

Test time adaptation (TTA) has emerged as a promising solution to adapt pre-trained models to new, unseen data distributions using unlabeled target domain data. However, most TTA methods are designed for independent data, often overlooking the time series data and rarely addressing forecasting tasks. This paper presents AdaNODEs, an innovative source-free TTA method tailored explicitly for time series forecasting. By leveraging Neural Ordinary Differential Equations (NODEs), we propose a novel adaptation framework that accommodates the unique characteristics of distribution shifts in time series data. Moreover, we innovatively propose a new loss function to tackle TTA for forecasting tasks. AdaNODEs only requires updating limited model parameters, showing effectiveness in capturing temporal dependencies while avoiding significant memory usage. Extensive experiments with one- and high-dimensional data demonstrate that AdaNODEs offer relative improvements of 5.88\% and 28.4\% over the SOTA baselines, especially demonstrating robustness across higher severity distribution shifts.
Paper Structure (13 sections, 3 equations, 4 figures, 1 table)

This paper contains 13 sections, 3 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Test-time Adaptation
  4. Neural Ordinary Differential Equations
  5. Method
  6. System Overview
  7. Adaptation in AdaNODEs
  8. Loss Function
  9. Experimental setup and results
  10. Experimental Setup
  11. Performance on One-dimensional Data
  12. Rotating MNIST
  13. Conclusion

Figures (4)

  • Figure 1: System overview. (a) Distribution shifts in time series forecasting. (b) AdaNODEs consists of an encoder, a latent NODEs, and a decoder. During TTA, additional parameters of $\alpha$ and $\gamma$ are incorporated and updated only. (c) The prediction at each time step $t$ is a distribution and the loss function is a combination of the negative log likelihood and KL divergence.
  • Figure 2: CCC for one-dimensional signals for severity levels L1-L5 for (a) amplitude or frequency change and (b) time delay.
  • Figure 3: Relative improvements of AdaNODEs over source model for all types of signals and severity levels.
  • Figure 4: Rotating MNIST with and without AdaNODEs at severity levels 1 and 5. AdaNODEs effectively adapts to match the slower rotation speed of the ground truth and extends the prediction over a longer sequence.