Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dependence Fidelity and Downstream Inference Stability in Generative Models

Nazia Riasat

Abstract

Recent advances in generative AI have led to increasingly realistic synthetic data, yet evaluation criteria remain focused on marginal distribution matching. While these diagnostics assess local realism, they provide limited insight into whether a generative model preserves the multivariate dependence structures governing downstream inference. We introduce covariance-level dependence fidelity as a practical criterion for evaluating whether a generative distribution preserves joint structure beyond univariate marginals. We establish three core results. First, distributions can match all univariate marginals exactly while exhibiting substantially different dependence structures, demonstrating marginal fidelity alone is insufficient. Second, dependence divergence induces quantitative instability in downstream inference, including sign reversals in regression coefficients despite identical marginal behavior. Third, explicit control of covariance-level dependence divergence ensures stable behavior for dependence-sensitive tasks such as principal component analysis. Synthetic constructions illustrate how dependence preservation failures lead to incorrect conclusions despite identical marginal distributions. These results highlight dependence fidelity as a useful diagnostic for evaluating generative models in dependence-sensitive downstream tasks, with implications for diffusion models and variational autoencoders. These guarantees apply specifically to procedures governed by covariance structure; tasks requiring higher-order dependence such as tail-event estimation require richer criteria.

Dependence Fidelity and Downstream Inference Stability in Generative Models

Abstract

Recent advances in generative AI have led to increasingly realistic synthetic data, yet evaluation criteria remain focused on marginal distribution matching. While these diagnostics assess local realism, they provide limited insight into whether a generative model preserves the multivariate dependence structures governing downstream inference. We introduce covariance-level dependence fidelity as a practical criterion for evaluating whether a generative distribution preserves joint structure beyond univariate marginals. We establish three core results. First, distributions can match all univariate marginals exactly while exhibiting substantially different dependence structures, demonstrating marginal fidelity alone is insufficient. Second, dependence divergence induces quantitative instability in downstream inference, including sign reversals in regression coefficients despite identical marginal behavior. Third, explicit control of covariance-level dependence divergence ensures stable behavior for dependence-sensitive tasks such as principal component analysis. Synthetic constructions illustrate how dependence preservation failures lead to incorrect conclusions despite identical marginal distributions. These results highlight dependence fidelity as a useful diagnostic for evaluating generative models in dependence-sensitive downstream tasks, with implications for diffusion models and variational autoencoders. These guarantees apply specifically to procedures governed by covariance structure; tasks requiring higher-order dependence such as tail-event estimation require richer criteria.
Paper Structure (64 sections, 3 theorems, 68 equations, 7 figures, 1 table)

This paper contains 64 sections, 3 theorems, 68 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Scope of the proposed criterion.
  3. Problem Setup and Notation
  4. Marginal fidelity.
  5. Dependence fidelity (covariance-level).
  6. Downstream functionals and inferential stability.
  7. Norm relations.
  8. Clarification on dependence notions.
  9. Related Work
  10. Evaluation of generative models.
  11. Distributional distances and integral probability metrics.
  12. Dependence modeling and copula theory.
  13. Stability of statistical procedures.
  14. Trustworthy and reliable AI.
  15. Main Results: Dependence Fidelity and Trustworthy Inference
  16. ...and 49 more sections

Key Result

Theorem 1

Let $d \geq 2$. There exist probability distributions $P$ and $Q$ on $\mathbb{R}^d$ such that: Moreover, the covariance divergence can be made arbitrarily large while maintaining exact marginal agreement nelsen2006sklar1959.

Figures (7)

  • Figure 1: Joint extreme-event probabilities for Gaussian and $t$ copulas. Despite identical marginals, the $t$ copula exhibits substantially higher joint tail risk due to heavy-tail dependence.
  • Figure 2: Covariance structures for two bivariate normal distributions with identical marginal distributions but opposite correlation signs. The change in dependence structure reverses the population regression slope, illustrating inferential instability under dependence divergence.
  • Figure 3: Empirical marginal CDFs for Gaussian and $t$-copula samples. The marginals are indistinguishable, demonstrating that marginal fidelity alone cannot detect dependence differences.
  • Figure 4: Population regression slope under two dependence structures with identical marginal distributions but opposite correlations. Dependence divergence induces a sign reversal in the regression coefficient, illustrating instability of downstream inference.
  • Figure 5: Distribution of Kolmogorov--Smirnov (KS) distances across genes comparing real and synthetic gene expression samples. Most KS values are small, indicating that the synthetic data approximately preserves univariate marginal distributions.
  • ...and 2 more figures

Theorems & Definitions (6)

  • Theorem 1: Marginal fidelity does not imply dependence fidelity
  • Theorem 2: Dependence divergence directly controls inferential sensitivity.
  • Theorem 3: Stability of PCA under covariance-level dependence fidelity
  • proof : Proof of Theorem 1
  • proof : Proof of Theorem 2
  • proof : Proof of Theorem 3