Diversified Scaling Inference in Time Series Foundation Models

TL;DR

Time Series Foundation Models (TSFMs) have advanced primarily through training-scale, but test-time compute remains underutilized. We formalize Diversified Scaling Inference by combining inference scaling (varying model size, context length, and temperature) with input perturbations that diversify the solution space, and show theoretically that diversified sampling expands the support from $\mathcal{S}_{\text{std}}$ to $\mathcal{S}_{\text{div}}$, yielding both asymptotic and finite-sample gains (notably a critical threshold $N^*$). We introduce RobustMSE as a budget-aware headroom metric and validate substantial improvements (up to ~50% in some settings) across TSFMs and datasets without retraining, elucidating when and how diversification provides gains. These findings offer practical guidance for deploying TSFMs efficiently in parallel environments, highlighting inference design as a powerful, compute-efficient alternative to further pre-training or fine-tuning.

Abstract

The advancement of Time Series Foundation Models (TSFMs) has been driven primarily by large-scale pre-training, but inference-time compute potential remains largely untapped. This work systematically investigates two questions: how do TSFMs behave under standard sampling-based inference scaling, and can controlled sampling diversity enhance performance? We first examine the properties of TSFMs under standard sampling often fail to adhere to scaling laws due to insufficient exploration of the solution space. Building on this, we then delve into diversified inference scaling via tailored time series perturbations to expand the generative distribution's support. We theoretically analyze the diversity-fidelity trade-off and derive a critical sample threshold for diversified sampling to outperform standard sampling. Extensive experiments across various TSFMs and datasets show proper diversified inference scaling yields substantial performance gains without parameter updates, establishing inference design as a critical, compute-efficient dimension of TSFM optimization. As an application, we propose RobustMSE, a rigorous metric to quantify the headroom performance of TSFM under a fixed budget. Overall, our findings clarify these factor interactions, enabling reliable performance via diverse large-scale inference time series in parallel environments without re-training TSFMs.

Figures (16)

• Figure 1: Illustration the relationship between scaling training hoffmann2022trainingcomputeoptimallargelanguageyao2024towards and diversified scaling inference in time series foundation models.
• Figure 2: Performance of Chronos on the data ETTh1 under different scaling factors and aggregation functions. We systematically evaluate the impact of three critical factors: model size, context length, and temperature. The number of sampling candidates is set to 128, plotted on a logarithmic axis. The stars indicate the initial convergence number to the minimum MSE for each configuration.
• Figure 3: Relationship between cosine similarity of time series samples and forecasting performance. Results are obtained on three datasets using Chronos-T5 models of different sizes. Solid lines represent performance under diversified sampling, with shaded regions indicating standard deviation across multiple runs. Horizontal lines correspond to the standard sampling for each model.
• Figure 4: Performance of Chronos on different datasets with effective perturbations. The dashed line represents the error of a single standard sampling, and results are reported under both EM and MV aggregators.
• Figure 5: Computation-Diversity.

Theorems & Definitions (5)

• Definition 4.1: Sampling Strategies
• Proposition 4.2: Asymptotic Lower Bound
• Proposition 4.3: Critical Sample Threshold
• Proposition 2.2: Asymptotic Lower Bound
• Proposition 2.3: Critical Sample Threshold