Trace Ratio vs Ratio Trace Methods for Multidimensional Dimensionality Reduction
Alaeddine Zahir, Franck Dufrenois, Khalide Jbilou, Ahmed Ratnani
TL;DR
This work develops a tensor-based extension of discriminant analysis by formulating Trace Ratio ($TR$) and Ratio Trace ($RT$) criteria in the Einstein product framework. It proves existence and uniqueness of TR solutions, derives optimality conditions, and introduces a Newton-type solver operating directly on tensors to preserve multilinear structure. The authors define Multilinear Discriminant Analysis via the Einstein product ($MDA_e$), with scatter tensors for between-class and within-class separability, and provide regularization and subspace-reduction strategies. Empirical results on MNIST, GTDB, DIV, and WDCM show that $MDA_e^{tr}$ and $MDA_e^{rt}$ outperform relevant matrix and tensor baselines, demonstrating robust, high-accuracy dimensionality reduction in high-dimensional, multi-modal data. The approach offers a scalable, structure-preserving alternative to flattening data and provides a principled link between discriminant analysis and tensor-based least-squares formulations.
Abstract
We propose a higher-order dimensionality reduction framework based on the Trace Ratio (TR) optimization problem. We establish conditions for existence and uniqueness of solutions and clarify the theoretical connection between the Trace Ratio and its surrogate, the Ratio Trace (RT) formulation. Building on these foundations, we design a Newton-type iterative algorithm that operates directly in the tensor domain via the Einstein product, avoiding data flattening and preserving multi-dimensional structure. This approach extends classical Linear Discriminant Analysis (LDA) to higher-order tensors, offering a natural generalization of trace-based dimensionality reduction from matrices to tensors. Numerical experiments on several benchmark datasets confirm the efficiency and robustness of the proposed methods, showing consistent improvements over existing matrix- and tensor-based techniques.