CSTS: A Benchmark for the Discovery of Correlation Structures in Time Series Clustering

TL;DR

CSTS tackles the challenge of objectively evaluating correlation-based time series clustering by introducing a synthetic, structure-first benchmark with ground-truth correlation patterns and controlled degradation. It provides two independent exploratory and confirmatory datasets across multiple data variants, with predefined degradation scenarios to disentangle algorithmic limitations from data effects. The framework demonstrates that correlation structures are largely robust to distribution shifts and sparsification but can be distorted by downsampling, especially for negative correlations, and that Spearman correlation offers superior preservation with a practical minimum of about $30$ observations per segment. A case study with TICC illustrates CSTS’s ability to diagnose distribution sensitivity and guide hyperparameter tuning, underscoring its value for rigorous, ground-truth-informed benchmarking in correlation-based time series clustering.

Abstract

Time series clustering promises to uncover hidden structural patterns in data with applications across healthcare, finance, industrial systems, and other critical domains. However, without validated ground truth information, researchers cannot objectively assess clustering quality or determine whether poor results stem from absent structures in the data, algorithmic limitations, or inappropriate validation methods, raising the question whether clustering is "more art than science" (Guyon et al., 2009). To address these challenges, we introduce CSTS (Correlation Structures in Time Series), a synthetic benchmark for evaluating the discovery of correlation structures in multivariate time series data. CSTS provides a clean benchmark that enables researchers to isolate and identify specific causes of clustering failures by differentiating between correlation structure deterioration and limitations of clustering algorithms and validation methods. Our contributions are: (1) a comprehensive benchmark for correlation structure discovery with distinct correlation structures, systematically varied data conditions, established performance thresholds, and recommended evaluation protocols; (2) empirical validation of correlation structure preservation showing moderate distortion from downsampling and minimal effects from distribution shifts and sparsification; and (3) an extensible data generation framework enabling structure-first clustering evaluation. A case study demonstrates CSTS's practical utility by identifying an algorithm's previously undocumented sensitivity to non-normal distributions, illustrating how the benchmark enables precise diagnosis of methodological limitations. CSTS advances rigorous evaluation standards for correlation-based time series clustering.

Figures (8)

• Figure 1: MAE distributions between target and empirical correlation structures across data variants. Lower values indicate better preservation of the original correlation structure.
• Figure 2: Effect of segment length on MAE between specified correlation structures and their estimation using different measures for the complete, non-normal data variant.
• Figure 3: Empirical distributions (blue) with theoretical PDF/PMF (red) for the complete variants (columns) and the three time series variates (rows). Q-Q plots in insets show quantile comparison between empirical and theoretical distributions. Raw and correlated variants show the standard normal distribution as theoretical distribution, non-normal and downsampled variants show the median parameters of the non-normal distributions.
• Figure 4: Empirical distributions (blue) with theoretical PDF/PMF (red) for the partial variants (70% observations) (columns) and the three time series variates (rows). Q-Q plots in insets show quantile comparison between empirical and theoretical distributions. Raw and correlated variants show the standard normal distribution as theoretical distribution, non-normal and downsampled variants show the median parameters of the non-normal distributions.
• Figure 5: Empirical distributions (blue) with theoretical PDF/PMF (red) for the sparse variants (10% observations) (columns) and the three time series variates (rows). Q-Q plots in insets show quantile comparison between empirical and theoretical distributions. Raw and correlated variants show the standard normal distribution as theoretical distribution, non-normal and downsampled variants show the median parameters of the non-normal distributions.