FFAD: A Novel Metric for Assessing Generated Time Series Data Utilizing Fourier Transform and Auto-encoder

TL;DR

FFAD addresses the lack of robust evaluation metrics for generated time-series by introducing a Fréchet-distance-based score computed in a learned latent space derived from a Fourier-domain representation. The approach preprocesses time-series via a Fourier transform to a fixed-length matrix $Z_D$ with shape $[N,m,2]$, trains a shared GRU Auto-Encoder to obtain latent representations $Y$, and defines the FFAD score as the Fréchet distance between the $Y$-distributions of two datasets: $ \text{FFAD Score} = ||\mu_{Y_{pos}} - \mu_{Y_{neg}}||^{2} + \mathrm{Tr}(\Sigma_{Y_{pos}}+\Sigma_{Y_{neg}}-2\sqrt{\Sigma_{Y_{pos}}\Sigma_{Y_{neg}}})$. The method is evaluated on large time-series collections (97 UCR datasets plus SWAN-SF) and demonstrates that FFAD can distinguish same-class from different-class samples, suggesting its value as a fundamental tool for evaluating generative time-series data and guiding future tasks such as flare forecasting. Overall, FFAD provides a practical, quantitatively-grounded pipeline for comparing real and synthetic time-series in a way that aligns with Fréchet-distance principles while leveraging frequency-domain preprocessing and deep latent representations.

Abstract

The success of deep learning-based generative models in producing realistic images, videos, and audios has led to a crucial consideration: how to effectively assess the quality of synthetic samples. While the Fréchet Inception Distance (FID) serves as the standard metric for evaluating generative models in image synthesis, a comparable metric for time series data is notably absent. This gap in assessment capabilities stems from the absence of a widely accepted feature vector extractor pre-trained on benchmark time series datasets. In addressing these challenges related to assessing the quality of time series, particularly in the context of Fréchet Distance, this work proposes a novel solution leveraging the Fourier transform and Auto-encoder, termed the Fréchet Fourier-transform Auto-encoder Distance (FFAD). Through our experimental results, we showcase the potential of FFAD for effectively distinguishing samples from different classes. This novel metric emerges as a fundamental tool for the evaluation of generative time series data, contributing to the ongoing efforts of enhancing assessment methodologies in the realm of deep learning-based generative models.

Figures (6)

• Figure 1: An example of Viewing a time signal in both the time and frequency domains utilizing Fourier Transform.
• Figure 2: Sub-figure (A) illustrates the procedure of employing Fourier Transformation as a preprocessing step for the original time series data, ensuring a consistent length for all datasets. Sub-figure (B) outlines the training procedure of the autoencoder.
• Figure 3: Shows the original time series and reconstructed time series utilizing different number of frequency components. This example is sourced from partition-1 of SWAN-SF.
• Figure 4: The results provide a comprehensive evaluation of the reconstruction performance on SWAN_SF partition-1.
• Figure 5: Shows the procedure of selecting the Auto-encoder model by calculating Mean Square Error (MSE) as the evaluation metric every 500 epochs, and identifies that the optimal model is achieved at the 3000th epoch.