Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

xTime: Extreme Event Prediction with Hierarchical Knowledge Distillation and Expert Fusion

Quan Li, Wenchao Yu, Suhang Wang, Minhua Lin, Lingwei Chen, Wei Cheng, Haifeng Chen

TL;DR

xTime tackles extreme-event forecasting in time series by coupling hierarchical knowledge distillation with a mixture-of-experts router. Each rarity level is handled by a dedicated expert trained through knowledge transfer from more common events, and a rarity-aware loss guides specialization; a wavelet-based multi-resolution decomposition enriches inputs, and a router fuses expert outputs in an ordinal, top-$k$ fashion. Empirical results across four real-world datasets show consistent improvements over strong baselines, including notable reductions in extreme-event prediction error with balanced use of intermediate information. The approach is scalable, flexible in rarity definitions, and demonstrates practical impact for proactive risk management in climate and healthcare domains.

Abstract

Extreme events frequently occur in real-world time series and often carry significant practical implications. In domains such as climate and healthcare, these events, such as floods, heatwaves, or acute medical episodes, can lead to serious consequences. Accurate forecasting of such events is therefore of substantial importance. Most existing time series forecasting models are optimized for overall performance within the prediction window, but often struggle to accurately predict extreme events, such as high temperatures or heart rate spikes. The main challenges are data imbalance and the neglect of valuable information contained in intermediate events that precede extreme events. In this paper, we propose xTime, a novel framework for extreme event forecasting in time series. xTime leverages knowledge distillation to transfer information from models trained on lower-rarity events, thereby improving prediction performance on rarer ones. In addition, we introduce a mixture of experts (MoE) mechanism that dynamically selects and fuses outputs from expert models across different rarity levels, which further improves the forecasting performance for extreme events. Experiments on multiple datasets show that xTime achieves consistent improvements, with forecasting accuracy on extreme events improving from 3% to 78%.

xTime: Extreme Event Prediction with Hierarchical Knowledge Distillation and Expert Fusion

TL;DR

xTime tackles extreme-event forecasting in time series by coupling hierarchical knowledge distillation with a mixture-of-experts router. Each rarity level is handled by a dedicated expert trained through knowledge transfer from more common events, and a rarity-aware loss guides specialization; a wavelet-based multi-resolution decomposition enriches inputs, and a router fuses expert outputs in an ordinal, top- fashion. Empirical results across four real-world datasets show consistent improvements over strong baselines, including notable reductions in extreme-event prediction error with balanced use of intermediate information. The approach is scalable, flexible in rarity definitions, and demonstrates practical impact for proactive risk management in climate and healthcare domains.

Abstract

Extreme events frequently occur in real-world time series and often carry significant practical implications. In domains such as climate and healthcare, these events, such as floods, heatwaves, or acute medical episodes, can lead to serious consequences. Accurate forecasting of such events is therefore of substantial importance. Most existing time series forecasting models are optimized for overall performance within the prediction window, but often struggle to accurately predict extreme events, such as high temperatures or heart rate spikes. The main challenges are data imbalance and the neglect of valuable information contained in intermediate events that precede extreme events. In this paper, we propose xTime, a novel framework for extreme event forecasting in time series. xTime leverages knowledge distillation to transfer information from models trained on lower-rarity events, thereby improving prediction performance on rarer ones. In addition, we introduce a mixture of experts (MoE) mechanism that dynamically selects and fuses outputs from expert models across different rarity levels, which further improves the forecasting performance for extreme events. Experiments on multiple datasets show that xTime achieves consistent improvements, with forecasting accuracy on extreme events improving from 3% to 78%.
Paper Structure (22 sections, 9 equations, 6 figures, 10 tables)

This paper contains 22 sections, 9 equations, 6 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Time Series Forecasting
  4. Large Time Series Foundation Models
  5. Extreme Event Prediction
  6. Problem Statement
  7. Proposed Method
  8. Expert Models
  9. Model Architecture
  10. Expert Model Training With Knowledge Distillation
  11. Hierarchical Rare Penalty Loss
  12. Router - Mixture of Expert Models
  13. Experiment Results
  14. Experimental Setup
  15. Datasets
  16. ...and 7 more sections

Figures (6)

  • Figure 1: The workflow of the proposed xTime framework: the input time series is first processed by pretrained expert models specialized for different rarity levels, and the router then fuses their outputs to generate the final prediction.
  • Figure 2: a) The training mechanism for rarity expert models and b) The detailed architecture for all expert models
  • Figure 3: Hierarchical rare penalty loss visualization
  • Figure 4: The training process for router
  • Figure 5: Hyper-parameters $\beta$ evaluation on different dataset: (a) Italy Air dataset (b) Beijing air dataset (c) Wind speed dataset (d) Wave height dataset. The letters M, V, and E in the figure denote the moderate rare mode, very rare model, and the extremely rare model, respectively.
  • ...and 1 more figures