Generating Synthetic Time Series Data for Cyber-Physical Systems

TL;DR

The paper addresses data augmentation for time-series in cyber-physical systems by proposing a pure transformer-based synthesizer trained as a conditional GAN and evaluated with the Wasserstein-Fourier Distance on normalized power spectral densities. It introduces a hierarchical transformer architecture with frequency-aware conditioning and a dual-head critic, and tests on the FEMTO/PRONOSTIA bearing dataset as well as a synthetic dataset. Results show the approach underperforms on real bearing data, revealing spectral misalignment despite plausible visuals, and similarly high divergence on synthetic data, highlighting the challenge of preserving long-range dependencies and spectral structure. The work motivates exploring diffusion-based methods and frequency-domain conditioning to improve synthetic TS quality for practical CPS data augmentation.

Abstract

Data augmentation is an important facilitator of deep learning applications in the time series domain. A gap is identified in the literature, demonstrating sparse exploration of the transformer, the dominant sequence model, for data augmentation in time series. A architecture hybridizing several successful priors is put forth and tested using a powerful time domain similarity metric. Results suggest the challenge of this domain, and several valuable directions for future work.

Figures (6)

• Figure 1: Each data point consists of a window (authentic signal) over time, and a label. The generator accepts a noise sample conditioned on the label, and the discriminator outputs evaluations of "realness" WGAN_origin, and a label to be compared to the target conditional.
• Figure 2: The normalized power spectrums WFD for the accelerometers of bearing 1 run 1 of the FEMTO dataset, taken at evenly spaced windows across the run. Blue indicates the beginning, red the end. While the shifts upwards in frequency are subtle in the first accelerometer, they are clear in the second. The depicted are not strict correspondents to actually observed frequencies, and so the lack of units on the x axis is intentional.
• Figure 3: While T, the number of time steps, is a power of 2, it takes $\text{log}_{2}(\text{T})$ blocks in the extraction pipeline to reduce each window to a tensor of one less rank. B is batch size and D channel depth. On the right is a break out of the employed transformer module.
• Figure 4: The seed dimensions (T-seed, D-seed) are set using hyper parameters. A smaller seed allows for more layers of up-scaling, allowing for greater learning capacity as the number of learned parameters increases. This also increases the computational cost of generation.
• Figure 5: The blue, green, and red windows are instances of the easy, medium, and hard artificial datasets. Each window has to temporally concurrent features, similar to the two accelerometer features in the FEMTO dataset. All windows are produced using compounded sine waves, and each class had a non-overlapping range of features used to stocastically generate each instance.