Accelerating Time Series Foundation Models with Speculative Decoding

TL;DR

This work targets the latency bottlenecks of autoregressive time-series foundation models used in web-scale forecasting. It introduces speculative decoding adapted to continuous time-series patches, employing a fast draft model and a larger target model to achieve substantial wall-clock speedups while preserving accuracy. The authors formalize both lossless and practical variants, derive speedup and compute formulas, prove deviation bounds, and provide an acceptance-estimation procedure with concentration guarantees for deployment. Empirical results across multiple benchmarks show speedups of up to around $2.2\times$ with modest accuracy trade-offs, and the approach requires no architectural changes to existing TSFMs, facilitating immediate deployment in production systems.

Abstract

Modern web applications--from real-time content recommendation and dynamic pricing to CDN optimization--increasingly rely on time-series forecasting to deliver personalized experiences to billions of users. Large-scale Transformer-based models have achieved state-of-the-art performance in time-series forecasting but suffer from high computational costs, limiting their deployment in latency-sensitive web applications. To address this challenge, we propose a general inference acceleration framework that adapts speculative decoding to autoregressive time-series models. Our approach employs a smaller "draft" model to propose future time-series patches, which are then verified in parallel by a larger "target" model, reducing the number of sequential forward passes required. We address key technical challenges in adapting this technique from discrete language tokens to continuous time-series distributions, including the design of acceptance criteria for multivariate Gaussian patches and practical variants that balance efficiency with accuracy. Through experiments on time series forecasting benchmarks relevant to web applications, we demonstrate significant inference speedups while maintaining competitive accuracy. The framework requires no architectural modifications to existing foundation models, making it immediately applicable to accelerate deployed time-series forecasting systems. Our implementation can be found at https://github.com/PranavSubbaraman/STRIDE

Figures (7)

• Figure 1: Speculative Decoding in web-scale forecasting: speedup for real-time recommendation and parallelization for CDN optimization to achieve low latency and high throughput.
• Figure 2: Speculative decoding for time-series patches. A lightweight draft model autoregressively proposes $\gamma$ candidate steps, while the target model validates $\gamma{+}1$ prefixes in a single batched pass. The process continues until the first rejection (marked ✗), yielding the longest accepted run $L$ and producing outputs $y_{1{:}L}$.
• Figure 3: Visualizing the acceptance rule $\alpha(x)=\min\{1,p(x\mid H)/q(x\mid H)\}$ for two Gaussian densities. The shaded overlap indicates the accepted portion of draft proposals.
• Figure 4: Illustrative accuracy vs. speed trade-off curves.
• Figure 5: Forecast comparison on a representative series segment. The speculative-decoding forecast nearly overlaps with the slower target-only baseline, indicating negligible accuracy loss.

Theorems & Definitions (35)

• Remark 1: Extension to general covariances
• Proposition 1: Dependence bounds
• proof
• Definition 1: Residual and overlap
• Definition 2: Lossless SD: single step
• Remark 2
• Theorem 1: Single-step exactness
• proof
• Theorem 2: Block & autoregressive exactness
• proof