Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Copula Entropy: Theory and Applications

Jian Ma

TL;DR

Copula Entropy (CE) reframes multivariate dependence through copula densities, proving CE is equivalent to mutual information and enabling a nonparametric estimation framework. The book develops CE theory, estimation methods, and a comprehensive CE-based methodology suite for structure learning, association/discovery, variable selection, causal inference via transfer entropy, and time-series analysis, including domain adaptation, normality tests, copula/hypothesis testing, and change-point detection. It further surveys extensive generalizations (Tsallis, Rényi, cumulative/copula variants) and a broad spectrum of real-world applications, arguing CE offers theoretical rigor, low computational cost, and practical versatility compared to kernel/dCor-based approaches. The work culminates in detailed evaluations, benchmarks, and practical implementations, highlighting CE as a unified, principled framework for measuring independence and causality across science and engineering. The results underscore CE’s robustness to nonlinear dependence, its margin-invariance, and its capacity to unify correlation with causal structure under a single theoretical umbrella.

Abstract

This is the monograph on the theory and applications of copula entropy (CE). This book first introduces the theory of CE, including its background, definition, theorems, properties, and estimation methods. The theoretical applications of CE to structure learning, association discovery, variable selection, causal discovery, system identification, time lag estimation, domain adaptation, multivariate normality test, copula hypothesis test, two-sample test, change point detection, and symmetry test are reviewed. The relationships between the theoretical applications and their connections to correlation and causality are discussed. The framework based on CE for measuring statistical independence and conditional independence is compared to the other similar ones. The advantages of CE based methodologies over the other comparable ones are evaluated with simulations. The mathematical generalizations of CE are reviewed. The real applications of CE to every branch of science and engineering are briefly introduced.

Copula Entropy: Theory and Applications

TL;DR

Copula Entropy (CE) reframes multivariate dependence through copula densities, proving CE is equivalent to mutual information and enabling a nonparametric estimation framework. The book develops CE theory, estimation methods, and a comprehensive CE-based methodology suite for structure learning, association/discovery, variable selection, causal inference via transfer entropy, and time-series analysis, including domain adaptation, normality tests, copula/hypothesis testing, and change-point detection. It further surveys extensive generalizations (Tsallis, Rényi, cumulative/copula variants) and a broad spectrum of real-world applications, arguing CE offers theoretical rigor, low computational cost, and practical versatility compared to kernel/dCor-based approaches. The work culminates in detailed evaluations, benchmarks, and practical implementations, highlighting CE as a unified, principled framework for measuring independence and causality across science and engineering. The results underscore CE’s robustness to nonlinear dependence, its margin-invariance, and its capacity to unify correlation with causal structure under a single theoretical umbrella.

Abstract

This is the monograph on the theory and applications of copula entropy (CE). This book first introduces the theory of CE, including its background, definition, theorems, properties, and estimation methods. The theoretical applications of CE to structure learning, association discovery, variable selection, causal discovery, system identification, time lag estimation, domain adaptation, multivariate normality test, copula hypothesis test, two-sample test, change point detection, and symmetry test are reviewed. The relationships between the theoretical applications and their connections to correlation and causality are discussed. The framework based on CE for measuring statistical independence and conditional independence is compared to the other similar ones. The advantages of CE based methodologies over the other comparable ones are evaluated with simulations. The mathematical generalizations of CE are reviewed. The real applications of CE to every branch of science and engineering are briefly introduced.

Paper Structure

This paper contains 262 sections, 12 theorems, 188 equations, 33 figures, 9 tables.

Table of Contents

  1. Preface
  2. Copula Entropy
  3. Background
  4. Copula theory
  5. Information theory
  6. Theory
  7. Properties
  8. Multivariate
  9. Properties of entropy
  10. Nonpositivity
  11. Invariance to monotonic transformation
  12. Marginal Free
  13. All orders
  14. Equivalence to correlation coefficient under Gaussian assumption
  15. Estimation
  16. ...and 247 more sections

Key Result

Theorem 1

sklar1959fonctions Given any $n$ dimensional random variables $\mathbf{X}$ with joint distribution $F(\mathbf{x})$ and margins $F_i(x_i)$. Then there exists a copula $C(\mathbf{u})$ such that If $F_i$ are continuous, then $C$ is unique. Conversely, if $C$ is a copula and $F_i$ are distribution functions, then the function $F$ defined by eq:sklar is a joint distribution with margins $F_i$.

Figures (33)

  • Figure 1: Results of structure learning experiments on real datasets.
  • Figure 2: Experimental results on the NHANES data with the six association measures.
  • Figure 3: Variables selected with three main dependence measures.
  • Figure 4: Causal strength from pressure to PM2.5 estimated with three measures.
  • Figure 5: Results of the experiment for identifying Lorenz system.
  • ...and 28 more figures

Theorems & Definitions (84)

  • Definition 1: Copula
  • Theorem 1: Sklar's theorem
  • Definition 2: Shannon entropy
  • Definition 3: Mutual Information
  • Theorem 2
  • Theorem 3
  • Definition 4: Conditional Mutual Information
  • Definition 5: Copula Entropy
  • Theorem 4
  • proof
  • ...and 74 more