Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Isometry pursuit

Samson Koelle, Marina Meila

TL;DR

Isometry pursuit is a convex algorithm for identifying isometric embeddings from within interpretable dictionaries that offers a synergistic alternative to greedy and brute force search for problems involving coordinate selection and diversification.

Abstract

Isometry pursuit is a convex algorithm for identifying orthonormal column-submatrices of wide matrices. It consists of a novel normalization method followed by multitask basis pursuit. Applied to Jacobians of putative coordinate functions, it helps identity isometric embeddings from within interpretable dictionaries. We provide theoretical and experimental results justifying this method. For problems involving coordinate selection and diversification, it offers a synergistic alternative to greedy and brute force search.

Isometry pursuit

TL;DR

Isometry pursuit is a convex algorithm for identifying isometric embeddings from within interpretable dictionaries that offers a synergistic alternative to greedy and brute force search for problems involving coordinate selection and diversification.

Abstract

Isometry pursuit is a convex algorithm for identifying orthonormal column-submatrices of wide matrices. It consists of a novel normalization method followed by multitask basis pursuit. Applied to Jacobians of putative coordinate functions, it helps identity isometric embeddings from within interpretable dictionaries. We provide theoretical and experimental results justifying this method. For problems involving coordinate selection and diversification, it offers a synergistic alternative to greedy and brute force search.

Paper Structure

This paper contains 24 sections, 7 theorems, 19 equations, 4 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Problem
  4. Interpretability and isometry
  5. Subset selection
  6. Method
  7. Ground truth
  8. Normalization
  9. Isometry pursuit
  10. Theory
  11. Two-stage isometry pursuit
  12. Experiments
  13. Discussion
  14. Supplement
  15. Algorithms
  16. ...and 9 more sections

Key Result

Proposition 1

The singular values $\sigma_1 \dots \sigma_D$ are equal to $1$ if and only if $U \in \mathbb{R}^{D \times D}$ is orthonormal.

Figures (4)

  • Figure 1: Plots of ground truth loss, normalized length, and basis pursuit loss for different values of $c$ in the one-dimensional case $D = 1$. The two losses are equivalent in the one-dimensional case.
  • Figure 2: Isometry losses $l_1$ for Wine, Iris, and Ethanol datasets across $R$ replicates. Lower brute losses are shown with turquoise, while lower two stage losses are shown with pink. Equal losses are shown with black lines. As detailed in Table \ref{['tab:experiments']}, losses are generally lower for two-stage isometry pursuit solutions.
  • Figure 3: Support Cardinalities for Wine, Iris, and Ethanol datasets
  • Figure 4: Comparison of Isometry and Group Lasso Losses across $25$ replicates for randomly downsampled Iris and Wine Datasets with $(P,D) = (4,15)$ and $(13, 18)$, respectively. Note that this further downsampling compared with Section \ref{['sec:experiments']} was necessary to compute global minimizers of BruteSearch. Lower brute losses are shown with turquoise, while lower two stage losses are shown with pink. Equal losses are shown with black lines.

Theorems & Definitions (10)

  • Definition 1
  • Definition 2
  • Proposition 1
  • Proposition 2
  • Definition 3: Symmetric normalization
  • Proposition 3
  • Proposition 4
  • Proposition 5
  • Proposition 6
  • Proposition 7