Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Early Classification of Time Series in Non-Stationary Cost Regimes

Aurélien Renault, Alexis Bondu, Antoine Cornuéjols, Vincent Lemaire

TL;DR

This paper investigates Early Classification of Time Series (ECTS) under deployment-time cost non-stationarity, including drift and stochastic costs. It develops online adaptations of representative separable ECTS methods that update only the trigger during deployment, exploring delayed, instant, and no-update regimes. Controlled experiments on a synthetic MNIST-1D dataset show that RL-based triggers offer robust performance across cost regimes, while non-myopic methods excel when updates are feasible. The findings emphasize the importance of adapting decision-time costs for reliable online ECTS, with open avenues for real-world datasets and joint handling of cost and data shifts.

Abstract

Early Classification of Time Series (ECTS) addresses decision-making problems in which predictions must be made as early as possible while maintaining high accuracy. Most existing ECTS methods assume that the time-dependent decision costs governing the learning objective are known, fixed, and correctly specified. In practice, however, these costs are often uncertain and may change over time, leading to mismatches between training-time and deployment-time objectives. In this paper, we study ECTS under two practically relevant forms of cost non-stationarity: drift in the balance between misclassification and decision delay costs, and stochastic realizations of decision costs that deviate from the nominal training-time model. To address these challenges, we revisit representative ECTS approaches and adapt them to an online learning setting. Focusing on separable methods, we update only the triggering model during deployment, while keeping the classifier fixed. We propose several online adaptations and baselines, including bandit-based and RL-based approaches, and conduct controlled experiments on synthetic data to systematically evaluate robustness under cost non-stationarity. Our results demonstrate that online learning can effectively improve the robustness of ECTS methods to cost drift, with RL-based strategies exhibiting strong and stable performance across varying cost regimes.

Early Classification of Time Series in Non-Stationary Cost Regimes

TL;DR

This paper investigates Early Classification of Time Series (ECTS) under deployment-time cost non-stationarity, including drift and stochastic costs. It develops online adaptations of representative separable ECTS methods that update only the trigger during deployment, exploring delayed, instant, and no-update regimes. Controlled experiments on a synthetic MNIST-1D dataset show that RL-based triggers offer robust performance across cost regimes, while non-myopic methods excel when updates are feasible. The findings emphasize the importance of adapting decision-time costs for reliable online ECTS, with open avenues for real-world datasets and joint handling of cost and data shifts.

Abstract

Early Classification of Time Series (ECTS) addresses decision-making problems in which predictions must be made as early as possible while maintaining high accuracy. Most existing ECTS methods assume that the time-dependent decision costs governing the learning objective are known, fixed, and correctly specified. In practice, however, these costs are often uncertain and may change over time, leading to mismatches between training-time and deployment-time objectives. In this paper, we study ECTS under two practically relevant forms of cost non-stationarity: drift in the balance between misclassification and decision delay costs, and stochastic realizations of decision costs that deviate from the nominal training-time model. To address these challenges, we revisit representative ECTS approaches and adapt them to an online learning setting. Focusing on separable methods, we update only the triggering model during deployment, while keeping the classifier fixed. We propose several online adaptations and baselines, including bandit-based and RL-based approaches, and conduct controlled experiments on synthetic data to systematically evaluate robustness under cost non-stationarity. Our results demonstrate that online learning can effectively improve the robustness of ECTS methods to cost drift, with RL-based strategies exhibiting strong and stable performance across varying cost regimes.
Paper Structure (24 sections, 6 equations, 3 figures, 1 table, 3 algorithms)

This paper contains 24 sections, 6 equations, 3 figures, 1 table, 3 algorithms.

Table of Contents

  1. Motivation
  2. Background
  3. Problem statement of standard ECTS
  4. Literature on standard ECTS
  5. Online ECTS with deployment-time cost mismatch
  6. Related domains
  7. Cost-Sensitive Learning
  8. Concept Drift in Data Streams
  9. Non-Stationary Reinforcement Learning and Bandits
  10. Bringing ECTS to Online Learning
  11. Delayed Updates
  12. Instant Updates
  13. No Updates (Cost-Agnostic Training)
  14. Experimental Protocol
  15. Evaluation Metrics
  16. ...and 9 more sections

Figures (3)

  • Figure 1: Normalized cumulated regret.
  • Figure 2: PV_D: Hold-out AvgCost, decomposed over both earliness and error rate axis. The thick AvgCost line corresponds to the $\alpha$ weighted sum of earliness and error rate. Purple curve describe the evolution of the $\alpha$ parameter.
  • Figure 3: Normalized cumulated regret. Costs of classes $\{1, 4, 7 \}$ (straight digits) are stochastic. Similar results were obtained with others noised classes. Averaged over five random seeds.