Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Algebraic Approach for Orthomax Rotations

Ryoya Fukasaku, Michio Yamamoto, Yutaro Kabata, Yasuhiko Ikematsu, Kei Hirose

TL;DR

This paper tackles the rotation step in factor analysis, specifically orthomax rotations, by introducing an algebraic framework that computes all equality-constrained stationary points of the orthomax criterion $Q_{\omega}(\Lambda)$ with $\Lambda=AT$ and $T^{\top}T=I_k$. By formulating the problem in polynomial rings and leveraging Lagrange multipliers, the authors derive an algebraic system whose all solutions yield global optima and stationary points independent of initialization, enabling exhaustive comparison across rotation criteria. Theoretical results include equivalent conditions for obtaining perfect simple structures and a Thurstone-simple-structure analysis, complemented by two algorithms that enumerate stationary points and classify them via bordered-Hessian tests. Monte Carlo simulations demonstrate that quartimax can yield more interpretable loadings than varimax, and that the algebraic approach can reveal stationary points that optimization-based methods may miss, though at higher computational cost. The work provides a principled, initialization-free framework for selecting interpretable factor loadings and points toward extensions to oblique rotations and broader algebraic analyses of rotation criteria.

Abstract

In exploratory factor analysis, rotation techniques are employed to derive interpretable factor loading matrices. Factor rotations deal with equality-constrained optimization problems aimed at determining a loading matrix based on measure of simplicity, such as ``perfect simple structure'' and ``Thurstone simple structure.'' Numerous criteria have been proposed, since the concept of simple structure is fundamentally ambiguous and involves multiple distinct aspects. However, most rotation criteria may fail to consistently yield a simple structure that is optimal for analytical purposes, primarily due to two challenges. First, existing optimization techniques, including the gradient projection descent method, exhibit strong dependence on initial values and frequently become trapped in suboptimal local optima. Second, multifaceted nature of simple structure complicates the ability of any single criterion to ensure interpretability across all aspects. In certain cases, even when a global optimum is achieved, other rotations may exhibit simpler structures in specific aspects. To address these issues, obtaining all equality-constrained stationary points -- including both global and local optima -- is advantageous. Fortunately, many rotation criteria are expressed as algebraic functions, and the constraints in the optimization problems in factor rotations are formulated as algebraic equations. Therefore, we can employ computational algebra techniques that utilize operations within polynomial rings to derive exact all equality-constrained stationary points. Unlike existing optimization methods, the computational algebraic approach can determine global optima and all stationary points, independent of initial values. We conduct Monte Carlo simulations to examine the properties of the orthomax rotation criteria, which generalizes various orthogonal rotation methods.

Algebraic Approach for Orthomax Rotations

TL;DR

This paper tackles the rotation step in factor analysis, specifically orthomax rotations, by introducing an algebraic framework that computes all equality-constrained stationary points of the orthomax criterion with and . By formulating the problem in polynomial rings and leveraging Lagrange multipliers, the authors derive an algebraic system whose all solutions yield global optima and stationary points independent of initialization, enabling exhaustive comparison across rotation criteria. Theoretical results include equivalent conditions for obtaining perfect simple structures and a Thurstone-simple-structure analysis, complemented by two algorithms that enumerate stationary points and classify them via bordered-Hessian tests. Monte Carlo simulations demonstrate that quartimax can yield more interpretable loadings than varimax, and that the algebraic approach can reveal stationary points that optimization-based methods may miss, though at higher computational cost. The work provides a principled, initialization-free framework for selecting interpretable factor loadings and points toward extensions to oblique rotations and broader algebraic analyses of rotation criteria.

Abstract

In exploratory factor analysis, rotation techniques are employed to derive interpretable factor loading matrices. Factor rotations deal with equality-constrained optimization problems aimed at determining a loading matrix based on measure of simplicity, such as ``perfect simple structure'' and ``Thurstone simple structure.'' Numerous criteria have been proposed, since the concept of simple structure is fundamentally ambiguous and involves multiple distinct aspects. However, most rotation criteria may fail to consistently yield a simple structure that is optimal for analytical purposes, primarily due to two challenges. First, existing optimization techniques, including the gradient projection descent method, exhibit strong dependence on initial values and frequently become trapped in suboptimal local optima. Second, multifaceted nature of simple structure complicates the ability of any single criterion to ensure interpretability across all aspects. In certain cases, even when a global optimum is achieved, other rotations may exhibit simpler structures in specific aspects. To address these issues, obtaining all equality-constrained stationary points -- including both global and local optima -- is advantageous. Fortunately, many rotation criteria are expressed as algebraic functions, and the constraints in the optimization problems in factor rotations are formulated as algebraic equations. Therefore, we can employ computational algebra techniques that utilize operations within polynomial rings to derive exact all equality-constrained stationary points. Unlike existing optimization methods, the computational algebraic approach can determine global optima and all stationary points, independent of initial values. We conduct Monte Carlo simulations to examine the properties of the orthomax rotation criteria, which generalizes various orthogonal rotation methods.
Paper Structure (12 sections, 10 theorems, 37 equations, 5 figures, 2 algorithms)

This paper contains 12 sections, 10 theorems, 37 equations, 5 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Orthomax rotations
  3. Factor rotations
  4. Orthogonal rotations
  5. Orthomax criteria
  6. Perfect simple structure
  7. Thurstone simple structure
  8. Algebraic approach for Orthomax rotations
  9. Monte Carlo Simulations
  10. Conclusion and future works
  11. Basic algebraic concepts
  12. Proof for Theorem \ref{['thm:thurstone-simple']}

Key Result

Proposition 1

The orthognal rotation criterion $Q$ has a stationary point at $T \in \mathbb{R}^{k \times k}$ restricted to $\{ T \in \mathbb{R}^{k \times k} : T^{\top} T = I_k\}$ if and only if $T^{\top} \frac{\partial Q(AT)}{\partial T}$ is a symmetric matrix, where denotes the gradient of $Q$ at $T = (t_{jl})^{1 \leq j \leq k}_{1 \leq l \leq k}$;

Figures (5)

  • Figure 1: rows such that the absolute values of two or more elements are less than $0.1$
  • Figure 2: rows such that the absolute values of one or more elements are less than $0.1$.
  • Figure 3: elements whose absolute values are less than $0.1$
  • Figure 4: Euclidean distances from the GPArotation outputs
  • Figure 5: second-order sufficient local maxima "max," second-order sufficient local minima "min," and second-order indeterminate point "indeterminate"

Theorems & Definitions (20)

  • Proposition 1
  • Proposition 2
  • Theorem 1
  • proof
  • Remark 1
  • Theorem 2
  • Remark 2
  • Corollary 1
  • Theorem 3
  • Definition 1
  • ...and 10 more