Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Copula Classifier: Theory, Consistency, and Empirical Evaluation

Agnideep Aich, Ashit Baran Aich

TL;DR

The paper introduces the Deep Copula Classifier (DCC), a generative, class-conditional model that decouples marginal estimation from dependence modeling using neural copula densities. It offers theoretical guarantees, including Bayes-consistency and a convergence rate of $O(n^{-r/(2r+d)})$ for $r$-smooth copulas, under standard regularity conditions. Empirically, DCC demonstrates Bayes-aligned decision regions in synthetic experiments and competitive, well-calibrated performance on the PIMA diabetes dataset, surpassing several baselines on ROC-AUC and achieving calibration rivaling logistic regression. The work highlights the practical and theoretical value of modeling feature dependencies with neural copulas, and discusses extensions to high-dimensional, semi-supervised, and streaming settings. Overall, DCC provides a principled, interpretable, and scalable alternative to independence-based classifiers for tasks where dependency structure is critical.

Abstract

We present the Deep Copula Classifier (DCC), a class-conditional generative model that separates marginal estimation from dependence modeling using neural copula densities. DCC is interpretable, Bayes-consistent, and achieves excess-risk $O(n^{-r/(2r+d)})$ for $r$-smooth copulas. In a controlled two-class study with strong dependence ($|ρ|=0.995$), DCC learns Bayes-aligned decision regions. With oracle or pooled marginals, it nearly reaches the best possible performance (accuracy $\approx 0.971$; ROC-AUC $\approx 0.998$). As expected, per-class KDE marginals perform less well (accuracy $0.873$; ROC-AUC $0.957$; PR-AUC $0.966$). On the Pima Indians Diabetes dataset, calibrated DCC ($τ=1$) achieves accuracy $0.879$, ROC-AUC $0.936$, and PR-AUC $0.870$, outperforming Logistic Regression, SVM (RBF), and Naive Bayes, and matching Logistic Regression on the lowest Expected Calibration Error (ECE). Random Forest is also competitive (accuracy $0.892$; ROC-AUC $0.933$; PR-AUC $0.880$). Directly modeling feature dependence yields strong, well-calibrated performance with a clear probabilistic interpretation, making DCC a practical, theoretically grounded alternative to independence-based classifiers.

Deep Copula Classifier: Theory, Consistency, and Empirical Evaluation

TL;DR

The paper introduces the Deep Copula Classifier (DCC), a generative, class-conditional model that decouples marginal estimation from dependence modeling using neural copula densities. It offers theoretical guarantees, including Bayes-consistency and a convergence rate of for -smooth copulas, under standard regularity conditions. Empirically, DCC demonstrates Bayes-aligned decision regions in synthetic experiments and competitive, well-calibrated performance on the PIMA diabetes dataset, surpassing several baselines on ROC-AUC and achieving calibration rivaling logistic regression. The work highlights the practical and theoretical value of modeling feature dependencies with neural copulas, and discusses extensions to high-dimensional, semi-supervised, and streaming settings. Overall, DCC provides a principled, interpretable, and scalable alternative to independence-based classifiers for tasks where dependency structure is critical.

Abstract

We present the Deep Copula Classifier (DCC), a class-conditional generative model that separates marginal estimation from dependence modeling using neural copula densities. DCC is interpretable, Bayes-consistent, and achieves excess-risk for -smooth copulas. In a controlled two-class study with strong dependence (), DCC learns Bayes-aligned decision regions. With oracle or pooled marginals, it nearly reaches the best possible performance (accuracy ; ROC-AUC ). As expected, per-class KDE marginals perform less well (accuracy ; ROC-AUC ; PR-AUC ). On the Pima Indians Diabetes dataset, calibrated DCC () achieves accuracy , ROC-AUC , and PR-AUC , outperforming Logistic Regression, SVM (RBF), and Naive Bayes, and matching Logistic Regression on the lowest Expected Calibration Error (ECE). Random Forest is also competitive (accuracy ; ROC-AUC ; PR-AUC ). Directly modeling feature dependence yields strong, well-calibrated performance with a clear probabilistic interpretation, making DCC a practical, theoretically grounded alternative to independence-based classifiers.

Paper Structure

This paper contains 64 sections, 5 theorems, 68 equations, 2 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Copula Theory and Foundations
  4. Generative Classification Paradigms
  5. Neural Copula Architectures
  6. Notation
  7. Basic objects.
  8. Distributions, expectations, probabilities.
  9. Class-conditional laws and marginals.
  10. Copulas.
  11. Neural copula parameterization (DCC).
  12. Decision rule and risks.
  13. Estimators.
  14. Smoothness and function classes.
  15. Norms and asymptotic notation.
  16. ...and 49 more sections

Key Result

Theorem 1

Let be the classifier produced by the Deep Copula Classifier (DCC) when all “hat” quantities (such as $\widehat{F}_{i\mid y}$, $\widehat{f}_{i\mid y}$, $\tilde{c}_y$, etc.) are estimated from $n$ i.i.d. training samples. In particular, $\widehat{Y}_{n}$ depends on $n$ because it uses the estimated class as outputs of Algorithm Algo1 trained on those $n$ samples. Define the Bayes-optimal decision

Figures (2)

  • Figure 1: Experiment 1 (synthetic). Top-left: data ($\rho=\pm0.995$). Top-right/bottom-left/bottom-right: DCC decision regions with oracle_normal, pooled KDE, and per_class KDE marginals.
  • Figure 2: PIMA (test, calibrated). Top: ROC (left) and Precision--Recall (right). Bottom: reliability diagram (ECE in legend).

Theorems & Definitions (7)

  • Definition 4.1: Copula
  • Definition 4.2: Deep Copula Classifier
  • Theorem 1: Consistency
  • Theorem 2: Convergence Rate
  • Proposition 4.1: Asymptotic copula validity
  • Corollary 4.1: Fast excess-risk rate under a multiclass margin
  • Lemma 1: Generative regret bound for plug-in MAP