Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Voronoi-based Column Selection Methods for Interpretable Dimensionality Reduction

Maria Emelianenko, Guy B. Oldaker

TL;DR

This work addresses interpretable dimensionality reduction via CUR decomposition by partitioning the CSSP into smaller, parallelizable tasks using Voronoi-based column partitioning. It introduces four partition-based methods built on CVOD and VQPCA, with adaptive variants that let the algorithm determine the number and size of partitions from data. Each partition is solved with a standard column-selector (DEIM) and then merged to form a full CUR approximation, with theoretical error bounds and empirical evidence showing competitive performance to DEIM on large datasets. The approach enables scalable, parallelizable column selection while preserving interpretability, and it lays groundwork for applications in model-order reduction and large-scale data analysis.

Abstract

In data analysis, there continues to be a need for interpretable dimensionality reduction methods whereby instrinic meaning associated with the data is retained in the reduced space. Standard approaches such as Principal Component Analysis (PCA) and the Singular Value Decomposition (SVD) fail at this task. A popular alternative is the CUR decomposition. In an SVD-like manner, the CUR decomposition approximates a matrix $A \in \mathbb{R}^{m \times n}$ as $A \approx CUR$, where $C$ and $R$ are matrices whose columns and rows are selected from the original matrix \cite{goreinov1997theory}, \cite{mahoney2009cur}. The difficulty in constructing a CUR decomposition is in determining which columns and rows to select when forming $C$ and $R$. Current column/row selection algorithms, particularly those that rely on an SVD, become infeasible as the size of the data becomes large \cite{dong2021simpler}. We address this problem by reducing the column/row selection problem to a collection of smaller sub-problems. The basic idea is to first partition the rows/columns of a matrix, and then apply an existing selection algorithm on each piece; for illustration purposes we use the Discrete Empirical Interpolation Method (\textsf{DEIM}) \cite{sorensen2016deim}. For the first task, we consider two existing algorithms that construct a Voronoi Tessellation (VT) of the rows and columns of a given matrix. We then extend these methods to automatically adapt to the data. The result is four data-driven row/column selection methods that are well-suited for parallelization, and compatible with nearly any existing column/row selection strategy. Theory and numerical examples show the design to be competitive with the original \textsf{DEIM} routine.

Adaptive Voronoi-based Column Selection Methods for Interpretable Dimensionality Reduction

TL;DR

This work addresses interpretable dimensionality reduction via CUR decomposition by partitioning the CSSP into smaller, parallelizable tasks using Voronoi-based column partitioning. It introduces four partition-based methods built on CVOD and VQPCA, with adaptive variants that let the algorithm determine the number and size of partitions from data. Each partition is solved with a standard column-selector (DEIM) and then merged to form a full CUR approximation, with theoretical error bounds and empirical evidence showing competitive performance to DEIM on large datasets. The approach enables scalable, parallelizable column selection while preserving interpretability, and it lays groundwork for applications in model-order reduction and large-scale data analysis.

Abstract

In data analysis, there continues to be a need for interpretable dimensionality reduction methods whereby instrinic meaning associated with the data is retained in the reduced space. Standard approaches such as Principal Component Analysis (PCA) and the Singular Value Decomposition (SVD) fail at this task. A popular alternative is the CUR decomposition. In an SVD-like manner, the CUR decomposition approximates a matrix as , where and are matrices whose columns and rows are selected from the original matrix \cite{goreinov1997theory}, \cite{mahoney2009cur}. The difficulty in constructing a CUR decomposition is in determining which columns and rows to select when forming and . Current column/row selection algorithms, particularly those that rely on an SVD, become infeasible as the size of the data becomes large \cite{dong2021simpler}. We address this problem by reducing the column/row selection problem to a collection of smaller sub-problems. The basic idea is to first partition the rows/columns of a matrix, and then apply an existing selection algorithm on each piece; for illustration purposes we use the Discrete Empirical Interpolation Method (\textsf{DEIM}) \cite{sorensen2016deim}. For the first task, we consider two existing algorithms that construct a Voronoi Tessellation (VT) of the rows and columns of a given matrix. We then extend these methods to automatically adapt to the data. The result is four data-driven row/column selection methods that are well-suited for parallelization, and compatible with nearly any existing column/row selection strategy. Theory and numerical examples show the design to be competitive with the original \textsf{DEIM} routine.
Paper Structure (13 sections, 3 theorems, 40 equations, 4 figures, 5 algorithms)

This paper contains 13 sections, 3 theorems, 40 equations, 4 figures, 5 algorithms.

Table of Contents

  1. Introduction
  2. The Column-Subset Selection Problem (CSSP)
  3. Notation
  4. Decomposing CSSP via Column Partitioning
  5. Review of CVOD
  6. Review of VQPCA
  7. Novel Adaptive Partitioning Strategies
  8. Adaptive CVOD
  9. Adaptive VQPCA
  10. Partitioned DEIM with Adaptive Column Selection
  11. Numerical Experiments
  12. Analysis and Discussion
  13. Conclusion

Key Result

Lemma 5.1

Let $A \in \mathbb{R}^{m \times n}$ with $\hbox{rank}(A) = \rho$, and let $C \in \mathbb{R}^{m \times r}$ be the result from applying PartionedDEIM to the output from any of the previously discussed partitioning algorithms. Then $C$ has full column rank.

Figures (4)

  • Figure 1: Normalized reconstruction errors for DEIM with and without partitioning on the sparse SNN1E3 data.
  • Figure 2: Normalized reconstruction errors for DEIM with and without partitioning on the MNIST data set using $k = 5,10$ Voronoi sets.
  • Figure 3: Evolution of the centroid dimensions for each Voronoi set resulting from adaptCVOD and adaptVQPCA applied to the MNIST data. In both cases, $k = 10$, $\hbox{rank} = 400$, and $\epsilon = 0.01$.
  • Figure 4: Energy profiles from all of the partition-based algorithms on the MNIST data set. The parameters used are $k = 10$, $\hbox{rank} = 400$, and $\epsilon = 0.01$.

Theorems & Definitions (6)

  • Lemma 5.1
  • proof
  • Lemma 7.1
  • proof
  • Theorem
  • proof