EntroPE: Entropy-Guided Dynamic Patch Encoder for Time Series Forecasting

TL;DR

Experiments across long-term forecasting benchmarks demonstrate that EntroPE improves both accuracy and efficiency, establishing entropy-guided dynamic patching as a promising new paradigm for time series modeling.

Abstract

Transformer-based models have significantly advanced time series forecasting, with patch-based input strategies offering efficiency and improved long-horizon modeling. Yet, existing approaches rely on temporally-agnostic patch construction, where arbitrary starting positions and fixed lengths fracture temporal coherence by splitting natural transitions across boundaries. This naive segmentation often disrupts short-term dependencies and weakens representation learning. In response, we propose EntroPE (Entropy-Guided Dynamic Patch Encoder), a novel, temporally informed framework that dynamically detects transition points via conditional entropy and dynamically places patch boundaries. This preserves temporal structure while retaining the computational benefits of patching. EntroPE consists of two key modules, namely an Entropy-based Dynamic Patcher (EDP) that applies information-theoretic criteria to locate natural temporal shifts and determine patch boundaries, and an Adaptive Patch Encoder (APE) that employs pooling and cross-attention to capture intra-patch dependencies and produce fixed-size latent representations. These embeddings are then processed by a global transformer to model inter-patch dynamics. Experiments across long-term forecasting benchmarks demonstrate that EntroPE improves both accuracy and efficiency, establishing entropy-guided dynamic patching as a promising new paradigm for time series modeling. Code is available at: https://github.com/Sachithx/EntroPE.

Figures (6)

• Figure 1: (a) Temporally-agnostic, and (b) Temporally-informed (proposed) patching with calculated entropies for ETTh1 sample.
• Figure 2: Comprehensive architecture of EntroPE. The model processes input through: (A) Entropy-Based Dynamic Patcher - A small causal transformer calculates entropy at each time point to identify boundaries where predictive uncertainty is high; (B) Adaptive Patch Encoder - Cross-attention layers aggregate intra-patch dependencies into fixed-size global embeddings; (C) Fusion Decoder - Cross-attention combines global patch context with local encoder hidden states for accurate forecasting.
• Figure 3: Ablation study for Dynamic and Static(patch_length) patching with single epoch duration.
• Figure 4: Ablation study for threshold sensitivity with epoch duration.
• Figure 5: EntroPE efficiency analysis.