Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Singular Learning Theory for Factor Analysis

Mathias Drton, Elizabeth Gross, Dimitra Kosta, Anton Leykin, Seth Sullivant, Daniel Windisch

TL;DR

The paper applies Watanabe's singular learning theory to factor analysis, focusing on real log canonical thresholds (RLCTs) as learning coefficients that govern marginal-likelihood asymptotics at singularities. It develops algebraic tools—fiber ideals, LQ-decomposition, Newton polyhedra, and blow-ups—to compute or bound RLCTs for the $k$-factor model, yielding a general upper bound $\ell_k(\Sigma_0)\le\frac{p(k+2)+r(p-k+1)}{4}$ under suitable rank conditions and exact values in diagonal and generic-one-factor cases. The work also analyzes the singularities of the one-factor model, showing distinct RLCTs across three strata and implications for sBIC-based model selection. Collectively, these results advance reliable Bayesian model comparison in latent-variable settings by quantifying how singular geometry affects marginal likelihood and by enabling sharper asymptotic penalties in factor-number selection.

Abstract

Watanabe's singular learning theory provides a framework for asymptotic analysis of Bayesian model selection for statistical models with singularities, where traditional statistical regularity assumptions fail. Learning coefficients, also known as real log canonical thresholds, play a central role in singular learning, as they govern the asymptotic behavior of Bayesian marginal likelihood integrals in settings where the Laplace approximations used for regular statistical models are not applicable. Learning coefficients are algebraic invariants that quantify the geometric complexity of a model and reveal how the singular structure impacts the model's generalization properties. In this paper, we apply algebraic methods to study the learning coefficients of factor analysis models, which are widely used latent variable models for continuously distributed data. Our main results provide a general upper bound for the learning coefficients as well as exact formulas for specific cases.

Singular Learning Theory for Factor Analysis

TL;DR

The paper applies Watanabe's singular learning theory to factor analysis, focusing on real log canonical thresholds (RLCTs) as learning coefficients that govern marginal-likelihood asymptotics at singularities. It develops algebraic tools—fiber ideals, LQ-decomposition, Newton polyhedra, and blow-ups—to compute or bound RLCTs for the -factor model, yielding a general upper bound under suitable rank conditions and exact values in diagonal and generic-one-factor cases. The work also analyzes the singularities of the one-factor model, showing distinct RLCTs across three strata and implications for sBIC-based model selection. Collectively, these results advance reliable Bayesian model comparison in latent-variable settings by quantifying how singular geometry affects marginal likelihood and by enabling sharper asymptotic penalties in factor-number selection.

Abstract

Watanabe's singular learning theory provides a framework for asymptotic analysis of Bayesian model selection for statistical models with singularities, where traditional statistical regularity assumptions fail. Learning coefficients, also known as real log canonical thresholds, play a central role in singular learning, as they govern the asymptotic behavior of Bayesian marginal likelihood integrals in settings where the Laplace approximations used for regular statistical models are not applicable. Learning coefficients are algebraic invariants that quantify the geometric complexity of a model and reveal how the singular structure impacts the model's generalization properties. In this paper, we apply algebraic methods to study the learning coefficients of factor analysis models, which are widely used latent variable models for continuously distributed data. Our main results provide a general upper bound for the learning coefficients as well as exact formulas for specific cases.

Paper Structure

This paper contains 16 sections, 18 theorems, 134 equations.

Table of Contents

  1. Introduction
  2. Preliminaries on real log canonical thresholds
  3. Real log canonical thresholds of ideals
  4. Newton polyhedra and monomial ideals
  5. Calculation rules
  6. Blow-up at a point
  7. Singular Learning Theory for factor analysis
  8. Marginal likelihood and learning coefficients
  9. Setup towards sBIC
  10. Fiber ideals for factor analysis
  11. LQ-decomposition
  12. Learning coefficients for factor analysis models
  13. Diagonal covariance matrices
  14. A general upper bound
  15. Generic one-factor covariance matrices
  16. ...and 1 more sections

Key Result

Theorem 1.1

Let $\ell_k(\Sigma_0)$ be the learning coefficient of the factor analysis model with $k$ latent factors at a fixed covariance matrix $\Sigma_0$ in this model. Let $r\in\{0,\dots,k\}$ be the minimum rank of any matrix $\Lambda\in\mathbb{R}^{p\times k}$ such that $\Sigma_0 = \psi + \Lambda\Lambda^T$ f

Theorems & Definitions (39)

  • Theorem 1.1
  • Proposition 1.2
  • Proposition 1.2
  • Example 2.4
  • Definition 3.1
  • Lemma 3.3
  • proof
  • Lemma 3.4
  • proof
  • proof
  • ...and 29 more