Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Causal Representation Meets Stochastic Modeling under Generic Geometry

Jiaxu Ren, Yixin Wang, Biwei Huang

TL;DR

This work addresses identifiability for latent causal representations when latent dynamics are continuous-time stochastic point processes, such as Hawkes processes. It develops a theory based on weak convergence and algebraic geometry, showing necessary and sufficient identifiability conditions under generic non-invertible mixing, for both linear and generic nonlinear mappings, using cumulants and Veronese embeddings. It introduces MUTATE, a time-adaptive variational autoencoder with a Neural PSD module to recover latent stochastic dynamics from high-dimensional observations, and provides testable conditions via interventions on the kernel dynamics. Across simulations and empirical data, MUTATE demonstrates strong latent recovery and causal structure identification, enabling scientific inferences about systems like genomics mutational accumulation and neuron spike mechanisms.

Abstract

Learning meaningful causal representations from observations has emerged as a crucial task for facilitating machine learning applications and driving scientific discoveries in fields such as climate science, biology, and physics. This process involves disentangling high-level latent variables and their causal relationships from low-level observations. Previous work in this area that achieves identifiability typically focuses on cases where the observations are either i.i.d. or follow a latent discrete-time process. Nevertheless, many real-world settings require identifying latent variables that are continuous-time stochastic processes (e.g., multivariate point processes). To this end, we develop identifiable causal representation learning for continuous-time latent stochastic point processes. We study its identifiability by analyzing the geometry of the parameter space. Furthermore, we develop MUTATE, an identifiable variational autoencoder framework with a time-adaptive transition module to infer stochastic dynamics. Across simulated and empirical studies, we find that MUTATE can effectively answer scientific questions, such as the accumulation of mutations in genomics and the mechanisms driving neuron spike triggers in response to time-varying dynamics.

Causal Representation Meets Stochastic Modeling under Generic Geometry

TL;DR

This work addresses identifiability for latent causal representations when latent dynamics are continuous-time stochastic point processes, such as Hawkes processes. It develops a theory based on weak convergence and algebraic geometry, showing necessary and sufficient identifiability conditions under generic non-invertible mixing, for both linear and generic nonlinear mappings, using cumulants and Veronese embeddings. It introduces MUTATE, a time-adaptive variational autoencoder with a Neural PSD module to recover latent stochastic dynamics from high-dimensional observations, and provides testable conditions via interventions on the kernel dynamics. Across simulations and empirical data, MUTATE demonstrates strong latent recovery and causal structure identification, enabling scientific inferences about systems like genomics mutational accumulation and neuron spike mechanisms.

Abstract

Learning meaningful causal representations from observations has emerged as a crucial task for facilitating machine learning applications and driving scientific discoveries in fields such as climate science, biology, and physics. This process involves disentangling high-level latent variables and their causal relationships from low-level observations. Previous work in this area that achieves identifiability typically focuses on cases where the observations are either i.i.d. or follow a latent discrete-time process. Nevertheless, many real-world settings require identifying latent variables that are continuous-time stochastic processes (e.g., multivariate point processes). To this end, we develop identifiable causal representation learning for continuous-time latent stochastic point processes. We study its identifiability by analyzing the geometry of the parameter space. Furthermore, we develop MUTATE, an identifiable variational autoencoder framework with a time-adaptive transition module to infer stochastic dynamics. Across simulated and empirical studies, we find that MUTATE can effectively answer scientific questions, such as the accumulation of mutations in genomics and the mechanisms driving neuron spike triggers in response to time-varying dynamics.
Paper Structure (73 sections, 27 theorems, 125 equations, 5 figures, 2 tables)

This paper contains 73 sections, 27 theorems, 125 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related Work
  4. Causal disentanglement and learning time series.
  5. Learning causal influences in stochastic processes.
  6. Problem Setup: Causal Representation with Stochastic Point Process
  7. Preliminaries and notations
  8. A Generative model for stochastic point processes
  9. Identifiability Theory
  10. Maximally identifiable equivalent classes
  11. Geometry characterization of model identifiability
  12. Identifying linear mixtures
  13. Identifying generic nonlinear equivalent classes
  14. The intuition of our theorem.
  15. From identifiability to testable conditions
  16. ...and 58 more sections

Key Result

Lemma 1

Let $N_t \in \mathbb{R}^p$ be a multivariate point process whose conditional intensity function $\lambda_t$ is governed by a convolution structure described in Eq. main:eq.2 and $\epsilon_t$ is a mean-zero and mutually independent noise. Then the intensity model admits the following weak convergence where the subscript $k$ denotes an arbitrary subsequence process and $\lambda_k^{\Delta}$ is the co

Figures (5)

  • Figure 3.1: Visualization of information loss in increasing filtration.
  • Figure 4.1: surfaces with generic hyperplanes (smooth gradient grid, 3D effect). (a) surface with one hyperplane;(b)Veronese with two hyperplanes;(c) surfaces with finite points
  • Figure C.1: Figure(a) is a kernel-delayed DAG $\mathscr{G}$ compatible with $\mathcal{M}_{\mathscr{G}}$; the system $\mathcal{M}_{\mathscr{G}}$ is a strict upper-triangular matrix. Figure (b) violates the topological order with additional edges $\{u_2\rightarrow u_1, v_3 \rightarrow v_2\}$.
  • Figure E.1: Visually Time-adaptive PSD Computation
  • Figure G.1: Connect three causal processes: revolution and degration of causal process. (a) classic autoregressive model that allows time-delayed causal influences. (b) causal process featured by a stochastic differential dynamics. (c) Hawkes process, a special self-exciting process.NE

Theorems & Definitions (46)

  • Definition 1: Conditional intensity, informal bacry_second_2014
  • Definition 2: Weakly-convergent equivalent class
  • Lemma 1: Bounding point process in Variational approximation
  • Lemma 2: Convergence to latent equivalent classes
  • Theorem 1: Linear identifiability of equivalent classes
  • Theorem 2: Fully nonlinear identifiability of equivalent classes
  • Theorem 3
  • Proposition 4.1: Almost surely bounded $\mathrm{dim}(I(V))$
  • Proposition 4.2: Positive possibility of bounded $\mathrm{dim}(I(V))$
  • Lemma A.1: Weak Convergence billingsley_convergence_1999-1
  • ...and 36 more