Closing the Gap Between Synthetic and Ground Truth Time Series Distributions via Neural Mapping

TL;DR

This work addresses fidelity gaps in vector-quantized time series generation by introducing NM-VQTSG, a U-Net–based neural mapper that refines synthetic outputs to better match ground-truth distributions. The mapper leverages stochastic vector quantization to approximate the target distribution and trains with an $L_1$ loss, selecting the optimal temperature $\tau$ via ROCKET-based FID minimization. Across 13 large UCR datasets, NM improves distributional fidelity (FID, IS, cFID) and yields visually and latently closer matches to real time series, demonstrating a robust post-hoc refinement applicable to any VQ-based TSG method. The approach offers practical impact by boosting realism of synthetic data, enabling more reliable downstream analysis and benchmarking in time series domains.

Abstract

In this paper, we introduce Neural Mapper for Vector Quantized Time Series Generator (NM-VQTSG), a novel method aimed at addressing fidelity challenges in vector quantized (VQ) time series generation. VQ-based methods, such as TimeVQVAE, have demonstrated success in generating time series but are hindered by two critical bottlenecks: information loss during compression into discrete latent spaces and deviations in the learned prior distribution from the ground truth distribution. These challenges result in synthetic time series with compromised fidelity and distributional accuracy. To overcome these limitations, NM-VQTSG leverages a U-Net-based neural mapping model to bridge the distributional gap between synthetic and ground truth time series. To be more specific, the model refines synthetic data by addressing artifacts introduced during generation, effectively aligning the distributions of synthetic and real data. Importantly, NM-VQTSG can be used for synthetic time series generated by any VQ-based generative method. We evaluate NM-VQTSG across diverse datasets from the UCR Time Series Classification archive, demonstrating its capability to consistently enhance fidelity in both unconditional and conditional generation tasks. The improvements are evidenced by significant improvements in FID, IS, and conditional FID, additionally backed up by visual inspection in a data space and a latent space. Our findings establish NM-VQTSG as a new method to improve the quality of synthetic time series. Our implementation is available on \url{https://github.com/ML4ITS/TimeVQVAE}.

Figures (6)

• Figure 1: Illustration of the two bottlenecks that cause a discrepancy between the distributions of GT samples $X$ and generated samples $\hat{X}$. $p(X)$, $p(\tilde{X})$, and $p(\hat{X})$ denote a distribution of GT samples $X$ (blue), reconstructed samples $\tilde{X}$ (orange), and generated samples $\hat{X}$ (green), respectively. Lastly, $p(X^\prime)$ represents a distribution of a stochastic variant of $X$ with stochastic vector quantization (red).
• Figure 2: Comparisons between $X$, $\hat{X}$, and $\hat{X}_R$. 200 randomly-sampled time series are plotted in overlap. For FordA and FaceAll, the differences between $X$, $\hat{X}$, and $\hat{X}_R$ are easily observable. For FreezerRegularTrain, TwoPatterns, and Wafer, the differences can be better detected in zoom-in, where $\hat{X}$ has somewhat noisy lines and those are corrected in $\hat{X}_R$.
• Figure 3: Comparisons between $\hat{x}$ and $\hat{x}_R$. For FaceAll and FordA, the magnitudes are corrected with addition of detailed patterns. For Freezer, a small downward bump at a timestep of around 240 in $\hat{x}$ is corrected in $\hat{x}_R$ (better viewed in zoom-in). For TwoPatterns and Wafer, $\hat{x}$'s horizontal lines are a bit wiggly and those lines are mapped to straight horizontal lines in $\hat{x}_R$ while retaining the rest of the patterns.
• Figure 4: Paired comparisons between $U$ and $\hat{U}$, as well as between $U$ and $\hat{U}_R$. $U$ denotes the representation from the FCN, computed as $U = \text{FCN}(X)$, and similarly for $\hat{U}$ and $\hat{U}_R$ by $\text{FCN}(\hat{X})$ and $\text{FCN}(\hat{X}_R)$, respectively. The representations originally have dimension size of 128, but we reduce it to 2 using PCA for the visualization.
• Figure 5: Experimental validation that our hypothesis of $p(\hat{X}) \approx p(\tilde{X}^\prime)$ is satisfied with an optimal $\tau$. The optimal $\tau$ is 1 and 0.1 for (a) and (b), respectively. $\hat{U}$ denotes the representation from ROCKET, computed as $\hat{U} = \text{ROCKET}(\hat{X})$, and similarly for $\tilde{U}^\prime$ by $\text{ROCKET}(\tilde{X}^\prime)$. The original dimension size of the representations is reduced to 2 using PCA for the visualization.

Theorems & Definitions (3)

• Theorem 3.1
• proof
• Claim 3.2