MoTM: Towards a Foundation Model for Time Series Imputation based on Continuous Modeling

TL;DR

This work tackles robust imputation of missing values $x_t$ in irregularly sampled time series under distribution shifts. It introduces MoTM, a foundation-model–like approach that builds a basis of TimeFlow implicit neural representations and uses a ridge orchestrator at inference to aggregate latent cues for zero-shot imputations across datasets with different sampling rates. Key contributions include pretraining a diverse basis of TimeFlow models, rapid per-series adaptation of latent codes, and a linear fusion mechanism that yields accurate imputations for any $t\\in[0,1]$ without retraining. Empirical results on synthetic and real-world data demonstrate strong ID and OOD generalization, competitive performance against supervised baselines, and efficient inference relative to retraining-based approaches, highlighting MoTM’s potential for scalable foundation imputation in heterogeneous time-series domains.

Abstract

Recent years have witnessed a growing interest for time series foundation models, with a strong emphasis on the forecasting task. Yet, the crucial task of out-of-domain imputation of missing values remains largely underexplored. We propose a first step to fill this gap by leveraging implicit neural representations (INRs). INRs model time series as continuous functions and naturally handle various missing data scenarios and sampling rates. While they have shown strong performance within specific distributions, they struggle under distribution shifts. To address this, we introduce MoTM (Mixture of Timeflow Models), a step toward a foundation model for time series imputation. Building on the idea that a new time series is a mixture of previously seen patterns, MoTM combines a basis of INRs, each trained independently on a distinct family of time series, with a ridge regressor that adapts to the observed context at inference. We demonstrate robust in-domain and out-of-domain generalization across diverse imputation scenarios (e.g., block and pointwise missingness, variable sampling rates), paving the way for adaptable foundation imputation models.

Figures (6)

• Figure 1: Illustration of the notations used in the rest of this paper.
• Figure 2: Plain INR vs Generalizable INR.
• Figure 3: Illustration of how the ridge orchestrator operates on a new time series $\mathbf{x^{(j)}}$, using a basis of two TimeFlow models. Note that the linear projection matrix $W^{(j)}$ is jointly optimized over all observed time steps of the new series $\mathbf{x^{(j)}}$.
• Figure 4: OOD ks1D1W dataset. MoTM performs imputation on (a) 70% missing timesteps and (b) three one-day missing blocks. Zoom on the first three weeks.
• Figure 5: OOD SpanishE dataset. MoTM imputation on four one-day missing blocks.