Revisiting PCA for time series reduction in temporal dimension

TL;DR

This paper revisits Principal Component Analysis (PCA) as a temporal-dimension reduction tool for time series analysis (TSA), challenging the belief that smoothing temporal structure harms sequential modeling. By applying PCA to sliding time-series windows of length $L$ and projecting onto the top $k$ components, the authors demonstrate that temporal compression can preserve accuracy while significantly reducing training and inference costs across Linear, Transformer-based (e.g., Informer, FEDformer), CNN, RNN, and patch-based (PatchTST) TSA models. Extensive experiments cover time series classification, forecasting, and extrinsic regression across 13 datasets, showing notable speedups (up to 40% for Informer) and memory reductions (up to 30% for TimesNet) with minimal or no loss in performance. The study also contrasts PCA with alternative reductions (shortening, downsampling, FFT/DWT, and reduction layers) and finds PCA consistently superior or competitive, supporting its adoption as a general preprocessing step in TSA pipelines.

Abstract

Revisiting PCA for Time Series Reduction in Temporal Dimension; Jiaxin Gao, Wenbo Hu, Yuntian Chen; Deep learning has significantly advanced time series analysis (TSA), enabling the extraction of complex patterns for tasks like classification, forecasting, and regression. Although dimensionality reduction has traditionally focused on the variable space-achieving notable success in minimizing data redundancy and computational complexity-less attention has been paid to reducing the temporal dimension. In this study, we revisit Principal Component Analysis (PCA), a classical dimensionality reduction technique, to explore its utility in temporal dimension reduction for time series data. It is generally thought that applying PCA to the temporal dimension would disrupt temporal dependencies, leading to limited exploration in this area. However, our theoretical analysis and extensive experiments demonstrate that applying PCA to sliding series windows not only maintains model performance, but also enhances computational efficiency. In auto-regressive forecasting, the temporal structure is partially preserved through windowing, and PCA is applied within these windows to denoise the time series while retaining their statistical information. By preprocessing time-series data with PCA, we reduce the temporal dimensionality before feeding it into TSA models such as Linear, Transformer, CNN, and RNN architectures. This approach accelerates training and inference and reduces resource consumption. Notably, PCA improves Informer training and inference speed by up to 40% and decreases GPU memory usage of TimesNet by 30%, without sacrificing model accuracy. Comparative analysis against other reduction methods further highlights the effectiveness of PCA in improving the efficiency of TSA models.

Figures (8)

• Figure 1: When PCA is applied to normal data, the order of data features is irrelevant, and there is no temporal correlation between features, as shown in (a); Our research demonstrates that PCA can also be applied to time series data, where the order of features (time steps) is significant and there is temporal correlation, as illustrated in (b).
• Figure 2: PCA is utilized for time series reduction in temporal dimension to enhance the efficiency of model training and inference in TSA.
• Figure 3: (a) PCA-inversed series. The PCA-inversed series is significantly smoother than the original series, indicating that PCA effectively filters out noise while preserving essential features. (b) Mean value distribution. The distributions of mean values for the original series and the PCA-reduced series show a high degree of overlap, demonstrating that PCA retains the key statistical characteristics.
• Figure 4: Training/inference time of various time series models with and without PCA preprocessing.
• Figure 5: GPU memory utilization of various time series models with and without PCA preprocessing.