Fréchet Cumulative Covariance Net for Deep Nonlinear Sufficient Dimension Reduction with Random Objects

TL;DR

The paper tackles nonlinear sufficient dimension reduction when responses are complex non-Euclidean objects. It introduces Fréchet Cumulative Covariance (FCCov) and a FCCov‑Net framework that uses neural networks (FNNs and ResNet‑CNNs) to learn nonlinear SDR directions, with unbiasedness at the $\sigma$-field level and nonasymptotic convergence guarantees. The authors establish an efficient $O(n^2\log n)$ FCCov computation, derive a tractable regularized objective, and provide sample-based estimators with explicit risk bounds for both Euclidean and metric-space responses. Extensive simulations show FCCov‑Net outperforms kernel and alternative deep SDR methods, and real data from JAFFE demonstrate robust, scalable performance with improved predictive and classification metrics. Overall, the work broadens nonlinear SDR to complex data types and offers strong theoretical and empirical performance guarantees with practical applicability.

Abstract

Nonlinear sufficient dimension reduction\citep{libing_generalSDR}, which constructs nonlinear low-dimensional representations to summarize essential features of high-dimensional data, is an important branch of representation learning. However, most existing methods are not applicable when the response variables are complex non-Euclidean random objects, which are frequently encountered in many recent statistical applications. In this paper, we introduce a new statistical dependence measure termed Fréchet Cumulative Covariance (FCCov) and develop a novel nonlinear SDR framework based on FCCov. Our approach is not only applicable to complex non-Euclidean data, but also exhibits robustness against outliers. We further incorporate Feedforward Neural Networks (FNNs) and Convolutional Neural Networks (CNNs) to estimate nonlinear sufficient directions in the sample level. Theoretically, we prove that our method with squared Frobenius norm regularization achieves unbiasedness at the $σ$-field level. Furthermore, we establish non-asymptotic convergence rates for our estimators based on FNNs and ResNet-type CNNs, which match the minimax rate of nonparametric regression up to logarithmic factors. Intensive simulation studies verify the performance of our methods in both Euclidean and non-Euclidean settings. We apply our method to facial expression recognition datasets and the results underscore more realistic and broader applicability of our proposal.

Figures (3)

• Figure 1: Typical facial expressions as predictors with emotion distributions as responses.
• Figure 2: Scatter plots of the first two sufficient predictors estimated by FCCov-Net.
• Figure 3: Scatter plots of the first three sufficient predictors estimated by FCCov-Net.

Theorems & Definitions (10)

• Definition 2.1: Fully-connected Neural Networks (FNNs)
• Definition 2.2: ResNet-type Convolutional Neural Networks(CNNs)
• Definition 3.1
• Theorem 3.2
• Theorem 3.3
• Theorem 4.1
• Remark 4.2
• Theorem 4.3
• Lemma 5.1
• Theorem 5.2