Efficient Data Reduction Via PCA-Guided Quantile Based Sampling

TL;DR

PCA-QS addresses the challenge of large-scale data reduction by combining PCA-based space partitioning with quantile-based sampling to produce representative, diverse subsamples. The method projects data onto the top $k$ principal components, partitions the transformed space into quantile groups, and samples from each group to preserve distributional structure while reducing data size. Theoretical analysis shows quantile distributions are scaled by the eigenvalues $\sqrt{\lambda_j}$, and empirical studies across synthetic and real datasets demonstrate improved distributional fidelity with competitive predictive performance and favorable computational efficiency. The framework also accommodates adaptive experimental design strategies (A-, D-, G-optimal and uncertainty-based) and clustering tasks, making PCA-QS a practical, interpretable tool for scalable statistical analysis.

Abstract

In large-scale statistical modeling, reducing data size through subsampling is essential for balancing computational efficiency and statistical accuracy. We propose a new method, Principal Component Analysis guided Quantile Sampling (PCA-QS), which projects data onto principal components and applies quantile-based sampling to retain representative and diverse subsets. Compared with uniform random sampling, leverage score sampling, and coreset methods, PCA-QS consistently achieves lower mean squared error and better preservation of key data characteristics, while also being computationally efficient. This approach is adaptable to a variety of data scenarios and shows strong potential for broad applications in statistical computing.

Figures (4)

• Figure 1: Comparison the "distances" between the retentive data sets of sampling methods to the original over data set generated with a gaussian-mixture data. Results shown here is based on 1000 replications under 10% sampling rate.
• Figure 2: Boxplots of model performance metrics across sampling methods. The Dynamic70 group is aggregated under the label Dyn, with the mean number of selected principal components = 266.25 (SD = 0.64). (Actual number of PCs used with dynamic pc option: mean: 266.25, standard deviation : 0.64, min: 264,'max: 268.) Runtime is reported with and without inclusion of the Coreset method due to its distinct computational cost.
• Figure 3: Distributions of performance metrics across 1,000 repetitions for different sampling methods.
• Figure 4: Distribution of Silhouette score of the differences between retentive and full datasets. The histogram illustrates the differences in silhouette scores between the retentive subset and the full dataset. Green dashed lines indicate the threshold at $\pm0.05$, covering 88.9% of cases, while red dashed lines indicate the threshold at $\pm0.1$, covering 99.5% of cases.

Theorems & Definitions (5)

• Remark 1
• Remark 2
• Remark 3: Topic for Future Investigation: High-Dimensional $p \sim n$ Case
• Remark 4: Historical and Statistical Foundations
• Remark 5