Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Amortized Predictability-aware Training Framework for Time Series Forecasting and Classification

Xu Zhang, Peng Wang, Yichen Li, Wei Wang

TL;DR

The paper tackles the challenge of training instability caused by low-predictability samples in time series data for both forecasting and classification. It introduces Amortized Predictability-aware Training Framework (APTF), which combines Hierarchical Predictability-aware Loss (HPL) with an amortization model to dynamically emphasize high-predictability samples while still leveraging noisy ones. The key contributions are (i) a hierarchical bucket-based loss that evolves with training to address predictability evolution, (ii) an amortization mechanism that reduces bias in predictability estimation, and (iii) extensive cross-task validation showing consistent improvements across 11 TSF and 128 TSC datasets with multiple baselines. The results demonstrate improved convergence, generalization, and robustness, with practical impact for noisy real-world time series such as finance and healthcare monitoring.

Abstract

Time series data are prone to noise in various domains, and training samples may contain low-predictability patterns that deviate from the normal data distribution, leading to training instability or convergence to poor local minima. Therefore, mitigating the adverse effects of low-predictability samples is crucial for time series analysis tasks such as time series forecasting (TSF) and time series classification (TSC). While many deep learning models have achieved promising performance, few consider how to identify and penalize low-predictability samples to improve model performance from the training perspective. To fill this gap, we propose a general Amortized Predictability-aware Training Framework (APTF) for both TSF and TSC. APTF introduces two key designs that enable the model to focus on high-predictability samples while still learning appropriately from low-predictability ones: (i) a Hierarchical Predictability-aware Loss (HPL) that dynamically identifies low-predictability samples and progressively expands their loss penalty as training evolves, and (ii) an amortization model that mitigates predictability estimation errors caused by model bias, further enhancing HPL's effectiveness. The code is available at https://github.com/Meteor-Stars/APTF.

Amortized Predictability-aware Training Framework for Time Series Forecasting and Classification

TL;DR

The paper tackles the challenge of training instability caused by low-predictability samples in time series data for both forecasting and classification. It introduces Amortized Predictability-aware Training Framework (APTF), which combines Hierarchical Predictability-aware Loss (HPL) with an amortization model to dynamically emphasize high-predictability samples while still leveraging noisy ones. The key contributions are (i) a hierarchical bucket-based loss that evolves with training to address predictability evolution, (ii) an amortization mechanism that reduces bias in predictability estimation, and (iii) extensive cross-task validation showing consistent improvements across 11 TSF and 128 TSC datasets with multiple baselines. The results demonstrate improved convergence, generalization, and robustness, with practical impact for noisy real-world time series such as finance and healthcare monitoring.

Abstract

Time series data are prone to noise in various domains, and training samples may contain low-predictability patterns that deviate from the normal data distribution, leading to training instability or convergence to poor local minima. Therefore, mitigating the adverse effects of low-predictability samples is crucial for time series analysis tasks such as time series forecasting (TSF) and time series classification (TSC). While many deep learning models have achieved promising performance, few consider how to identify and penalize low-predictability samples to improve model performance from the training perspective. To fill this gap, we propose a general Amortized Predictability-aware Training Framework (APTF) for both TSF and TSC. APTF introduces two key designs that enable the model to focus on high-predictability samples while still learning appropriately from low-predictability ones: (i) a Hierarchical Predictability-aware Loss (HPL) that dynamically identifies low-predictability samples and progressively expands their loss penalty as training evolves, and (ii) an amortization model that mitigates predictability estimation errors caused by model bias, further enhancing HPL's effectiveness. The code is available at https://github.com/Meteor-Stars/APTF.
Paper Structure (35 sections, 2 equations, 7 figures, 13 tables, 2 algorithms)

This paper contains 35 sections, 2 equations, 7 figures, 13 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Amortized predictability-aware training framework (APTF)
  4. Hierarchical predictability-aware loss (HPL)
  5. Low-predictability to high-predictability buckets
  6. Loss weights for paying more attention to high predictability samples
  7. Predictability-aware loss with predictability evolution
  8. Hierarchical Predictability-aware Loss with predictability evolution
  9. Amortized hierarchical predictability-aware loss
  10. Experiments and results
  11. Experimental settings
  12. Results of time series forecasting (TSF) and time series classification (TSC)
  13. Short-term forecasting
  14. Long-term forecasting
  15. Time series classification
  16. ...and 20 more sections

Figures (7)

  • Figure 1: The proposed general training framework APTF for time series forecasting and classification.
  • Figure 2: Illustration of different bucketing strategies for computing the predictability-aware loss. (a) A fixed number of buckets is used throughout training, ignoring the predictability evolution issue. (b) The number of buckets decreases as training progresses, addressing predictability evolution but discarding previous bucket partitions. (c) Hierarchical buckets consider both predictability evolution and previous bucket partitioning strategies.
  • Figure 3: Loss landscapes of Autoformer with and without APTF on the Fund sales dataset for 10-step forecasting. A flatter minimum (larger blue area) indicates better generalization. Additional cases are provided in Appendix Figure \ref{['fig:full_res_loss_landscape']}.
  • Figure 4: Hyperparameter sensitivity analysis.
  • Figure 5: Time series visualization of Fund sales dataset.
  • ...and 2 more figures