DDT: A Dual-Masking Dual-Expert Transformer for Energy Time-Series Forecasting

TL;DR

DDT tackles energy time-series forecasting by addressing data heterogeneity and the tension between strict causality and adaptive feature selection. It merges a Dual-Masking mechanism (combining $M_{ ext{causal}}$ and $M_{ ext{dynamic}}$) with a Dual-Expert Transformer whose temporal and cross-variable dynamics are fused by a dynamic gate, and augments this with time-series patching and a configurable CI mode. The approach yields state-of-the-art results across multiple energy benchmarks and horizons, with ablations confirming the complementary roles of the masking components and the dual-expert design. The work provides practical, scalable improvements for grid management and renewable integration, and points toward further gains in interpretability and edge deployment.

Abstract

Accurate energy time-series forecasting is crucial for ensuring grid stability and promoting the integration of renewable energy, yet it faces significant challenges from complex temporal dependencies and the heterogeneity of multi-source data. To address these issues, we propose DDT, a novel and robust deep learning framework for high-precision time-series forecasting. At its core, DDT introduces two key innovations. First, we design a dual-masking mechanism that synergistically combines a strict causal mask with a data-driven dynamic mask. This novel design ensures theoretical causal consistency while adaptively focusing on the most salient historical information, overcoming the rigidity of traditional masking techniques. Second, our architecture features a dual-expert system that decouples the modeling of temporal dynamics and cross-variable correlations into parallel, specialized pathways, which are then intelligently integrated through a dynamic gated fusion module. We conducted extensive experiments on 7 challenging energy benchmark datasets, including ETTh, Electricity, and Solar. The results demonstrate that DDT consistently outperforms strong state-of-the-art baselines across all prediction horizons, establishing a new benchmark for the task.

Figures (7)

• Figure 1: Schematic diagram of model time series prediction
• Figure 2: Schematic diagram of the structure of the DDT model
• Figure 3: Multivariate Feature Fusion Mechanism
• Figure 4: Causal Mask Mechanism
• Figure 5: Dynamic Causal Mask Mechanism