DropoutTS: Sample-Adaptive Dropout for Robust Time Series Forecasting

TL;DR

DropoutTS introduces a capacity-centric approach to robustness in time series forecasting by adaptively modulating learning capacity per sample using a spectral noise score. Through a Spectral Noise Scorer and a differentiable Sample-Adaptive Dropout, it maps per-sample noise to dropout rates, enabling end-to-end optimization without architectural changes. The method relies on spectral sparsity to distinguish dominant signal components from noise, employing global detrending, log-scale spectral normalization, an SFM-anchored spectral filter, and residual reconstruction to generate a proxy-free noise metric. Across seven real-world datasets and the Synth-12 benchmark, DropoutTS delivers consistent gains across diverse backbones, mitigates the Fixed Dropout Paradox, and reduces training time while preserving or improving inference latency. The results demonstrate the practicality of a universal, plug-in robustness enhancer for real-time forecasting tasks in finance, climate, healthcare, and industry.

Abstract

Deep time series models are vulnerable to noisy data ubiquitous in real-world applications. Existing robustness strategies either prune data or rely on costly prior quantification, failing to balance effectiveness and efficiency. In this paper, we introduce DropoutTS, a model-agnostic plugin that shifts the paradigm from "what" to learn to "how much" to learn. DropoutTS employs a Sample-Adaptive Dropout mechanism: leveraging spectral sparsity to efficiently quantify instance-level noise via reconstruction residuals, it dynamically calibrates model learning capacity by mapping noise to adaptive dropout rates - selectively suppressing spurious fluctuations while preserving fine-grained fidelity. Extensive experiments across diverse noise regimes and open benchmarks show DropoutTS consistently boosts superior backbones' performance, delivering advanced robustness with negligible parameter overhead and no architectural modifications. Our code is available at https://github.com/CityMind-Lab/DropoutTS.

Figures (14)

• Figure 1: Comparison of robustness paradigms. 1) Data-Centric Selection (Data-Level): Binary pruning causes inevitable information loss. 2) Prior-Centric Modeling (Latent-Level): Rigid probabilistic constraints raise complexity. 3) Capacity-Centric Modulation (Function-Level): Dynamically calibrates capacity via sample-adaptive dropout to balance fidelity and robustness.
• Figure 2: Signal Regimes and Noise Profiles.Top: Clean signal ranging from stationary to non-stationarity in mean (Trend), frequency (Chirp), and variance (AM). Bottom: Noise profiles modeling aleatoric uncertainty (Gaussian), epistemic anomalies (Heavy-tail), and observation failures (Missing Values).
• Figure 3: Validation of Spectral Sparsity. Analysis of a composite signal (Periodic + Trend + Chirp + AM) under varying noise. Left: Corrupted temporal inputs. Middle: Frequency spectra exhibit distinct separation between sparse high-energy signal and dispersed noise. Right: Spectral thresholding reconstruction matches ground truth, verifying robustness to outliers/missing data.
• Figure 4: Overview of DropoutTS. A model-agnostic plugin replacing default dropout with adaptive dropout, consisting of two components: (1) Spectral Noise Scorer, which quantifies instance-level noise via spectral residual reconstruction; (2) Sample-Adaptive Dropout, which maps noise scores to dynamic dropout rates and generates differentiable masks for end-to-end learning.
• Figure 5: Resolution of the Fixed Dropout Paradox. (a) Informer exhibits erratic, non-monotonic error trajectories under fixed dropout (Red). (b) Crossformer reveals a counter-intuitive inversion where baseline error decreases as noise increases. DropoutTS (Blue) rectifies these anomalies, restoring rational monotonicity and ensuring optimal performance on clean signals.

Theorems & Definitions (2)

• Theorem 6.1: Sub-optimality of Fixed Dropout
• Theorem 6.2: Adaptive Generalization Bound