Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TRACE: A Generalizable Drift Detector for Streaming Data-Driven Optimization

Yuan-Ting Zhong, Ting Huang, Xiaolin Xiao, Yue-Jiao Gong

TL;DR

TRACE addresses unknown concept drift in streaming data for SDDO by introducing a transferable drift estimator that tokenizes data streams into statistical sequences and learns drift patterns with a dual-attention Transformer. The model, comprising a Sequence Embedding Module, G-MSA and C-MSA, and a pointer-like drift head, is trained on synthetic drift scenarios to generalize to unseen environments. TRACE-EA demonstrates plug-and-play integration with SDDEAs, leveraging an archive for knowledge transfer to swiftly adapt optimization after drift. Across diverse benchmarks and real-world tasks, TRACE shows superior drift detection accuracy and improved SDDO performance, validating its generalization and practical applicability.

Abstract

Many optimization tasks involve streaming data with unknown concept drifts, posing a significant challenge as Streaming Data-Driven Optimization (SDDO). Existing methods, while leveraging surrogate model approximation and historical knowledge transfer, are often under restrictive assumptions such as fixed drift intervals and fully environmental observability, limiting their adaptability to diverse dynamic environments. We propose TRACE, a TRAnsferable C}oncept-drift Estimator that effectively detects distributional changes in streaming data with varying time scales. TRACE leverages a principled tokenization strategy to extract statistical features from data streams and models drift patterns using attention-based sequence learning, enabling accurate detection on unseen datasets and highlighting the transferability of learned drift patterns. Further, we showcase TRACE's plug-and-play nature by integrating it into a streaming optimizer, facilitating adaptive optimization under unknown drifts. Comprehensive experimental results on diverse benchmarks demonstrate the superior generalization, robustness, and effectiveness of our approach in SDDO scenarios.

TRACE: A Generalizable Drift Detector for Streaming Data-Driven Optimization

TL;DR

TRACE addresses unknown concept drift in streaming data for SDDO by introducing a transferable drift estimator that tokenizes data streams into statistical sequences and learns drift patterns with a dual-attention Transformer. The model, comprising a Sequence Embedding Module, G-MSA and C-MSA, and a pointer-like drift head, is trained on synthetic drift scenarios to generalize to unseen environments. TRACE-EA demonstrates plug-and-play integration with SDDEAs, leveraging an archive for knowledge transfer to swiftly adapt optimization after drift. Across diverse benchmarks and real-world tasks, TRACE shows superior drift detection accuracy and improved SDDO performance, validating its generalization and practical applicability.

Abstract

Many optimization tasks involve streaming data with unknown concept drifts, posing a significant challenge as Streaming Data-Driven Optimization (SDDO). Existing methods, while leveraging surrogate model approximation and historical knowledge transfer, are often under restrictive assumptions such as fixed drift intervals and fully environmental observability, limiting their adaptability to diverse dynamic environments. We propose TRACE, a TRAnsferable C}oncept-drift Estimator that effectively detects distributional changes in streaming data with varying time scales. TRACE leverages a principled tokenization strategy to extract statistical features from data streams and models drift patterns using attention-based sequence learning, enabling accurate detection on unseen datasets and highlighting the transferability of learned drift patterns. Further, we showcase TRACE's plug-and-play nature by integrating it into a streaming optimizer, facilitating adaptive optimization under unknown drifts. Comprehensive experimental results on diverse benchmarks demonstrate the superior generalization, robustness, and effectiveness of our approach in SDDO scenarios.

Paper Structure

This paper contains 27 sections, 2 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Stream Tokenization for Drift Modeling
  5. TRACE for Learning Transferable Drift Patterns
  6. Sequence Embedding.
  7. The TRACE Encoder.
  8. Drift Classification Head.
  9. Training for Transferability.
  10. Integration into Streaming Optimizers
  11. The TRACE-EA Framework.
  12. The Detection-Adaptation Loop.
  13. Experiments
  14. Experimental Setup
  15. Training setup.
  16. ...and 12 more sections

Figures (7)

  • Figure 1: TRACE offers a learnable and generalizable approach to drift detection in the SDDO landscape.
  • Figure 2: TRACE Architecture. The upper part illustrates the overall framework, while the lower part details its key components: the Sequence Embedding Module; the Dual-Attention Encoder; and the Drift Classification Head.
  • Figure 3: Detector performance comparison. See Appendix D-1 for additional results.
  • Figure 4: Convergence trajectories over the first 20 environments. Left: SDDObench_F4D4; Right: DBG_F1D2.
  • Figure 5: Attention distribution learned by C-MSA. Higher weights are assigned to tokens near drift time point and the first context token, with red '$\times$' indicating TRACE's predicted drift and the ground truth at token 12.
  • ...and 2 more figures