Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantify the Causes of Causal Emergence: Critical Conditions of Uncertainty and Asymmetry in Causal Structure

Liye Jia, Fengyufan Yang, Ka Lok Man, Erick Purwanto, Sheng-Uei Guan, Jeremy Smith, Yutao Yue

TL;DR

It is suggested that the redistribution of uncertainties for coarse-graining is the cause of causal emergence and the thresholds that determine if CE occurs or not are analyzed.

Abstract

Beneficial to advanced computing devices, models with massive parameters are increasingly employed to extract more information to enhance the precision in describing and predicting the patterns of objective systems. This phenomenon is particularly pronounced in research domains associated with deep learning. However, investigations of causal relationships based on statistical and informational theories have posed an interesting and valuable challenge to large-scale models in the recent decade. Macroscopic models with fewer parameters can outperform their microscopic counterparts with more parameters in effectively representing the system. This valuable situation is called "Causal Emergence." This paper introduces a quantification framework, according to the Effective Information and Transition Probability Matrix, for assessing numerical conditions of Causal Emergence as theoretical constraints of its occurrence. Specifically, our results quantitatively prove the cause of Causal Emergence. By a particular coarse-graining strategy, optimizing uncertainty and asymmetry within the model's causal structure is significantly more influential than losing maximum information due to variations in model scales. Moreover, by delving into the potential exhibited by Partial Information Decomposition and Deep Learning networks in the study of Causal Emergence, we discuss potential application scenarios where our quantification framework could play a role in future investigations of Causal Emergence.

Quantify the Causes of Causal Emergence: Critical Conditions of Uncertainty and Asymmetry in Causal Structure

TL;DR

It is suggested that the redistribution of uncertainties for coarse-graining is the cause of causal emergence and the thresholds that determine if CE occurs or not are analyzed.

Abstract

Beneficial to advanced computing devices, models with massive parameters are increasingly employed to extract more information to enhance the precision in describing and predicting the patterns of objective systems. This phenomenon is particularly pronounced in research domains associated with deep learning. However, investigations of causal relationships based on statistical and informational theories have posed an interesting and valuable challenge to large-scale models in the recent decade. Macroscopic models with fewer parameters can outperform their microscopic counterparts with more parameters in effectively representing the system. This valuable situation is called "Causal Emergence." This paper introduces a quantification framework, according to the Effective Information and Transition Probability Matrix, for assessing numerical conditions of Causal Emergence as theoretical constraints of its occurrence. Specifically, our results quantitatively prove the cause of Causal Emergence. By a particular coarse-graining strategy, optimizing uncertainty and asymmetry within the model's causal structure is significantly more influential than losing maximum information due to variations in model scales. Moreover, by delving into the potential exhibited by Partial Information Decomposition and Deep Learning networks in the study of Causal Emergence, we discuss potential application scenarios where our quantification framework could play a role in future investigations of Causal Emergence.
Paper Structure (18 sections, 19 equations, 27 figures, 3 tables, 7 algorithms)

This paper contains 18 sections, 19 equations, 27 figures, 3 tables, 7 algorithms.

Table of Contents

  1. Introduction
  2. Information-based Causal Emergency Theories
  3. Effective Information (EI), Transition Probabilistic Matrix (TPM), and Causal Emergence (CE)
  4. Our Scope and Reasons
  5. Quantifying the critical conditions of the CE: The framework
  6. The design of our framework: Three hypotheses
  7. Relationship between $determinism$ and $x$ and Conditions' Quantification Equation (CQE)
  8. Quantifying the critical conditions of CE: The reduction of CMs' uncertainty
  9. Quantifying the critical conditions of CE: The necessity of considerations about asymmetry decrement
  10. Tried Extensions of Quantification Framework and Possible Improvements
  11. Potential Application: The Design of Coarse-graining Strategies for CE
  12. Potential Application: Neural Networks for Predicting CE Conditions
  13. Conclusions
  14. Implementations of the Generation Process of Synthetic TPMs
  15. Derivations of Mathematical Relationship between $x$ and $determinism$
  16. ...and 3 more sections

Figures (27)

  • Figure 1: Deterministic, Stochastic, and Degenerate State Dynamics and TPMs of Three Systems. Both State Dynamics and TPMs show that mechanism of system to transfer current states $S_t=\{00,\ 01,\ 10,\ 11\}$ into future states $S^F$ following conditional probabilities in every row of TPMs. While the distribution of current states satisfies the MED, TPMs become numerical representations of causal structures of systems or models.
  • Figure 2: The example of CE while coarse-graining a CM_m with four states into a CM_M with two states hoel2017map.
  • Figure 3: The overview of three hypotheses to construct three corresponding algorithms within Appendix \ref{['algorithms_frame']} as our framework's module for generating the synthetic TPMs to study the critical CE conditions.
  • Figure 4: As the asymmetry determines the increment of the CM's $degeneracy$, this characteristic can independently influence the EI of the CM without any uncertainty. However, the values of the CM's asymmetry (measured by the $degeneracy$) can also be affected by the uncertainty (measured by the decrement of $determinism$).
  • Figure 5: Implement the $deg\_vector=[1,\ 3]$ to transform a symmetric model into the asymmetric.
  • ...and 22 more figures