On relative universality, regression operator, and conditional independence

TL;DR

The paper addresses how conditional independence between $X$ and $Y$ can be fully captured by the regression operator through a refined notion of relative universality. It introduces $ε$-measurability to weaken the original strong relative universality and proves that, under compactness and density/completeness assumptions, the range of the regression operator $R_{XY}=\Sigma_{XX}^{\dagger}\Sigma_{XY}$ matches the central class ${\frak S}_{Y|X}$, yielding unbiasedness and Fisher consistency for GSIR. A key contribution is clarifying a gap in Li (2018) by replacing measurability with $ε$-measurability to ensure rigorous density arguments; this links the regression operator to conditional independence via the central $\,\sigma$-field ${\cal G}_{Y|X}$. The results extend beyond nonlinear SDR to a general framework for conditional independence in statistics and machine learning, with implications for graphical models, probability embedding, causal inference, and Bayesian estimation. Practically, the theory supports regression-based methods that estimate conditional independence structures through covariance-operator machinery in RKHS settings.

Abstract

The notion of relative universality with respect to a σ-field was introduced to establish the unbiasedness and Fisher consistency of an estimator in nonlinear sufficient dimension reduction. However, there is a gap in the proof of this result in the existing literature. The existing definition of relative universality seems to be too strong for the proof to be valid. In this note we modify the definition of relative universality using the concept of ǫ-measurability, and rigorously establish the mentioned unbiasedness and Fisher consistency. The significance of this result is beyond its original context of sufficient dimension reduction, because relative universality allows us to use the regression operator to fully characterize conditional independence, a crucially important statistical relation that sits at the core of many areas and methodologies in statistics and machine learning, such as dimension reduction, graphical models, probability embedding, causal inference, and Bayesian estimation.