Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SOFARI-R: High-Dimensional Manifold-Based Inference for Latent Responses

Zemin Zheng, Xin Zhou, Jinchi Lv

TL;DR

This work develops SOFARI-R, a high-dimensional, manifold-based framework for statistical inference on latent right singular vectors in multi-task regression, addressing the asymmetry between left and right SVD components.Two variants are proposed: SOFARI-R$_s$ for strongly orthogonal factors and a general SOFARI-R for weakly orthogonal factors, both leveraging Neyman near-orthogonality on Stiefel manifolds to obtain bias-corrected, asymptotically normal estimators without requiring an inverse Hessian in the strong-orthogonality setting.Theoretical results establish asymptotic normality and consistent variance estimation in both regimes, with simulation and real-data analyses (FRED-MD) demonstrating accurate inference and interpretable latent response structure across three layers.Overall, SOFARI-R extends the SOFARI framework to latent responses, enabling inference on all significant eigenvectors in high-dimensional multi-task learning and offering practical tools for predictor- and response-focused latent factor analysis.

Abstract

Data reduction with uncertainty quantification plays a key role in various multi-task learning applications, where large numbers of responses and features are present. To this end, a general framework of high-dimensional manifold-based SOFAR inference (SOFARI) was introduced recently in Zheng, Zhou, Fan and Lv (2024) for interpretable multi-task learning inference focusing on the left factor vectors and singular values exploiting the latent singular value decomposition (SVD) structure. Yet, designing a valid inference procedure on the latent right factor vectors is not straightforward from that of the left ones and can be even more challenging due to asymmetry of left and right singular vectors in the response matrix. To tackle these issues, in this paper we suggest a new method of high-dimensional manifold-based SOFAR inference for latent responses (SOFARI-R), where two variants of SOFARI-R are introduced. The first variant deals with strongly orthogonal factors by coupling left singular vectors with the design matrix and then appropriately rescaling them to generate new Stiefel manifolds. The second variant handles the more general weakly orthogonal factors by employing the hard-thresholded SOFARI estimates and delicately incorporating approximation errors into the distribution. Both variants produce bias-corrected estimators for the latent right factor vectors that enjoy asymptotically normal distributions with justified asymptotic variance estimates. We demonstrate the effectiveness of the newly suggested method using extensive simulation studies and an economic application.

SOFARI-R: High-Dimensional Manifold-Based Inference for Latent Responses

TL;DR

This work develops SOFARI-R, a high-dimensional, manifold-based framework for statistical inference on latent right singular vectors in multi-task regression, addressing the asymmetry between left and right SVD components.Two variants are proposed: SOFARI-R$_s$ for strongly orthogonal factors and a general SOFARI-R for weakly orthogonal factors, both leveraging Neyman near-orthogonality on Stiefel manifolds to obtain bias-corrected, asymptotically normal estimators without requiring an inverse Hessian in the strong-orthogonality setting.Theoretical results establish asymptotic normality and consistent variance estimation in both regimes, with simulation and real-data analyses (FRED-MD) demonstrating accurate inference and interpretable latent response structure across three layers.Overall, SOFARI-R extends the SOFARI framework to latent responses, enabling inference on all significant eigenvectors in high-dimensional multi-task learning and offering practical tools for predictor- and response-focused latent factor analysis.

Abstract

Data reduction with uncertainty quantification plays a key role in various multi-task learning applications, where large numbers of responses and features are present. To this end, a general framework of high-dimensional manifold-based SOFAR inference (SOFARI) was introduced recently in Zheng, Zhou, Fan and Lv (2024) for interpretable multi-task learning inference focusing on the left factor vectors and singular values exploiting the latent singular value decomposition (SVD) structure. Yet, designing a valid inference procedure on the latent right factor vectors is not straightforward from that of the left ones and can be even more challenging due to asymmetry of left and right singular vectors in the response matrix. To tackle these issues, in this paper we suggest a new method of high-dimensional manifold-based SOFAR inference for latent responses (SOFARI-R), where two variants of SOFARI-R are introduced. The first variant deals with strongly orthogonal factors by coupling left singular vectors with the design matrix and then appropriately rescaling them to generate new Stiefel manifolds. The second variant handles the more general weakly orthogonal factors by employing the hard-thresholded SOFARI estimates and delicately incorporating approximation errors into the distribution. Both variants produce bias-corrected estimators for the latent right factor vectors that enjoy asymptotically normal distributions with justified asymptotic variance estimates. We demonstrate the effectiveness of the newly suggested method using extensive simulation studies and an economic application.

Paper Structure

This paper contains 44 sections, 26 theorems, 412 equations, 2 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. High-dimensional manifold-based SOFAR inference for latent responses
  3. SOFARI-Rs under strongly orthogonal factors
  4. SOFARI-R under weakly orthogonal factors
  5. Asymptotic properties
  6. Technical conditions
  7. Asymptotic theory of SOFARI-Rs
  8. Asymptotic theory of SOFARI-R
  9. Simulation studies
  10. Real data application
  11. Discussions
  12. Proofs of Theorems \ref{['theo:strong:vk']}--\ref{['theo:weak:var']} and Propositions \ref{['prop:deri2']}--\ref{['prop:strong3:w']}
  13. Proof of Theorem \ref{['theo:strong:vk']}
  14. Proof of Theorem \ref{['theo:strong:var']}
  15. Proof of Theorem \ref{['theo:weak:uk']}
  16. ...and 29 more sections

Key Result

Proposition 1

Under constraint SVDc, the orthonormal vectors ${\boldsymbol{u}}_i$ with $1 \leq i \leq r^*$ belong to the Stiefel manifold $\operatorname{St}(1,n) = \{\boldsymbol{u} \in \mathbb{R}^n : \boldsymbol{u}^T\boldsymbol{u} = 1 \}$. The gradient of $\widetilde{\psi}_k$ on the manifold is ${\bf Q}(\frac{\

Figures (2)

  • Figure 1: The kernel density estimates (KDEs) for the distributions of SOFARI-R estimates on the latent right factor vectors in different sparse SVD layers against the target standard normal density based on $1000$ replications for setting 1 in Section \ref{['new.Sec.5']}. Left panel: the KDEs of $\mathrm{T}_{1,1}, \mathrm{T}_{2,4}$, and $\mathrm{T}_{3,7}$; right panel: the KDEs of $\mathrm{T}_{1,15}, \mathrm{T}_{2,15}$, and $\mathrm{T}_{3,15}$, all viewed from top to bottom. The blue curves represent the KDEs for SOFARI-R estimators, whereas the red curves stand for the target standard normal density.
  • Figure 2: Bar charts of the significant components in the top three latent right factor vectors (i.e., the right singular vectors weighted by the corresponding variance-adjusted singular values). The significant components correspond to ones listed in Table 5. Different colors represent the three factors, F1, F2, and F3, ranked by their estimated singular values, respectively. The y-axis indicates the magnitude and sign of the corresponding coefficient in each factor. The dashed vertical lines indicate the boundaries between different groups of variables.

Theorems & Definitions (29)

  • Proposition 1
  • Proposition 2
  • Proposition 3
  • Proposition 4
  • Definition 1: SOFAR SVD estimates
  • Definition 2: Approximate Inverse
  • Theorem 1
  • Definition 3
  • Theorem 2
  • Theorem 3
  • ...and 19 more