Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalized Eigenvalue Problems with Generative Priors

Zhaoqiang Liu, Wen Li, Junren Chen

TL;DR

This work studies GEPs under generative priors, assuming that the underlying leading generalized eigenvector lies within the range of a Lipschitz continuous generative model, and proposes an iterative algorithm called the Projected Rayleigh Flow Method (PRFM) to approximate the optimal solution.

Abstract

Generalized eigenvalue problems (GEPs) find applications in various fields of science and engineering. For example, principal component analysis, Fisher's discriminant analysis, and canonical correlation analysis are specific instances of GEPs and are widely used in statistical data processing. In this work, we study GEPs under generative priors, assuming that the underlying leading generalized eigenvector lies within the range of a Lipschitz continuous generative model. Under appropriate conditions, we show that any optimal solution to the corresponding optimization problems attains the optimal statistical rate. Moreover, from a computational perspective, we propose an iterative algorithm called the Projected Rayleigh Flow Method (PRFM) to approximate the optimal solution. We theoretically demonstrate that under suitable assumptions, PRFM converges linearly to an estimated vector that achieves the optimal statistical rate. Numerical results are provided to demonstrate the effectiveness of the proposed method.

Generalized Eigenvalue Problems with Generative Priors

TL;DR

This work studies GEPs under generative priors, assuming that the underlying leading generalized eigenvector lies within the range of a Lipschitz continuous generative model, and proposes an iterative algorithm called the Projected Rayleigh Flow Method (PRFM) to approximate the optimal solution.

Abstract

Generalized eigenvalue problems (GEPs) find applications in various fields of science and engineering. For example, principal component analysis, Fisher's discriminant analysis, and canonical correlation analysis are specific instances of GEPs and are widely used in statistical data processing. In this work, we study GEPs under generative priors, assuming that the underlying leading generalized eigenvector lies within the range of a Lipschitz continuous generative model. Under appropriate conditions, we show that any optimal solution to the corresponding optimization problems attains the optimal statistical rate. Moreover, from a computational perspective, we propose an iterative algorithm called the Projected Rayleigh Flow Method (PRFM) to approximate the optimal solution. We theoretically demonstrate that under suitable assumptions, PRFM converges linearly to an estimated vector that achieves the optimal statistical rate. Numerical results are provided to demonstrate the effectiveness of the proposed method.

Paper Structure

This paper contains 21 sections, 6 theorems, 79 equations, 4 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contributions
  4. Notation
  5. Preliminary
  6. Setup and Main Assumptions
  7. Discussion on the Technical Novelty Compared to liu2022generative and tan2018sparse
  8. Main Results
  9. Experiments
  10. Experiment Setup
  11. Results for GGEPs
  12. Conclusion and Future Work
  13. Instances of GGEPs
  14. Proof of Theorem \ref{['thm:main_thm1']} (Guarantees for Globally Optimal Solutions)
  15. Auxiliary Lemmas for the Proof of Theorem \ref{['thm:main_thm1']}
  16. ...and 6 more sections

Key Result

Theorem 3.1

Suppose that Assumptions assp:normG, assp:generalized_eigens, and assp:pert hold for the GEP and generative model $G$. Let $\hat{\mathbf{u}}$ be a globally optimal solution to Eq. eq:opt_ggep for GGEP. Then, for any $\delta \in (0,1)$ satisfying $\delta = O((k \log \frac{4Lr}{\delta})/n)$, when $m = where $C_1$ is a positive constant depending on $\mathbf{A}$ and $\mathbf{B}$.

Figures (4)

  • Figure 1: Reconstructed images of the MNIST dataset for $(\hat{\mathbf{A}},\hat{\mathbf{B}})$ generated from Eqs. \ref{['eq:nonID_tildeB']} and \ref{['eq:PR_tildeAB']}.
  • Figure 2: Quantitative results of the performance of the methods on MNIST.
  • Figure 3: Reconstructed images of the CelebA dataset for $(\hat{\mathbf{A}},\hat{\mathbf{B}})$ generated from Eqs. \ref{['eq:nonID_tildeB']} and \ref{['eq:PR_tildeAB']}.
  • Figure 4: Quantitative results of the performance of the methods on CelebA.

Theorems & Definitions (15)

  • Remark 2.2
  • Remark 2.5
  • Theorem 3.1
  • Remark 3.2
  • Theorem 3.3
  • Remark 3.4
  • Remark 3.5
  • Lemma B.1
  • proof
  • Lemma C.1
  • ...and 5 more