Longitudinal Causal Image Synthesis

TL;DR

A tabular-visual causal graph (TVCG) for CLIS is established through a novel integration of generative imaging, continuous-time modeling, and structural causal models combined with a neural network, substantiating the reliability and utility in clinics.

Abstract

Clinical decision-making relies heavily on causal reasoning and longitudinal analysis. For example, for a patient with Alzheimer's disease (AD), how will the brain grey matter atrophy in a year if intervened on the A-beta level in cerebrospinal fluid? The answer is fundamental to diagnosis and follow-up treatment. However, this kind of inquiry involves counterfactual medical images which can not be acquired by instrumental or correlation-based image synthesis models. Yet, such queries require counterfactual medical images, not obtainable through standard image synthesis models. Hence, a causal longitudinal image synthesis (CLIS) method, enabling the synthesis of such images, is highly valuable. However, building a CLIS model confronts three primary yet unmet challenges: mismatched dimensionality between high-dimensional images and low-dimensional tabular variables, inconsistent collection intervals of follow-up data, and inadequate causal modeling capability of existing causal graph methods for image data. In this paper, we established a tabular-visual causal graph (TVCG) for CLIS overcoming these challenges through a novel integration of generative imaging, continuous-time modeling, and structural causal models combined with a neural network. We train our CLIS based on the ADNI dataset and evaluate it on two other AD datasets, which illustrate the outstanding yet controllable quality of the synthesized images and the contributions of synthesized MRI to the characterization of AD progression, substantiating the reliability and utility in clinics.

Figures (12)

• Figure 1: The illustration of causal and longitudinal disease progression. Along the time axis, the accumulation of misfolded proteins has a causal effect on brain atrophy. Multiple covariates, including gene type, sex, education level, and baseline age, also have causal effects on the disease trajectories (shown in yellow). The blue trajectory represents the counterfactual brain atrophy when successfully intervened on the misfolded protein level at $t_1$, which deviates from the fact and results in a different MRI at $t_2$. We attempt to develop a computational model that leads to such a success.
• Figure 2: An illustration of (a) a window causal graph of three variables in two-time steps. (b) the corresponding deduced summary causal graph.
• Figure 3: The training and inference phases of the proposed tabular-visual causal graph (TVCG). The training includes two parts. (1) The tabular-only causal graph construction. Firstly causal discovery algorithms are applied to the observed tabular variables with assumptions and prior medical knowledge. Then for the recovered edges, the generative function is fitted. (2) The tabular-visual causal graph (TOCG) construction. An Intervened MRI Synthesis Module (ISM) is trained for the MRI generative. ISM is severed as an edge between tabular volume variables and MRI thus building TVCG over TOCG. The ISM training is a 3-step process: training an image generator, an image encoder, and an intervened latent generator. During inference, a set of baseline and optional intervention variables are input into the trained TVCG. The model computes the intervened tabular variables using the TOCG within TVCG, and the ISM generates the intervened MRI. These synthesized results are then applicable to downstream classification tasks.
• Figure 4: The recovered causal graph, represented by a summary causal graph. Refer to Table \ref{['Shared Edges']} for a precise reference to each edge.
• Figure 5: The p-values of edges that do not meet the criteria of t-test in at least one dataset. Each sample of ADNI includes two adjacent session data as $(\boldsymbol{X}^{T}, \boldsymbol{X}^{T+1})$. The NACC dataset lacks multiple session data for a single subject thus only edges of instantaneous causality can be tested.

Theorems & Definitions (4)

• Definition 1: Window Causal Graph
• Definition 2: Causal discovery
• Definition 3: Structural Causal Model
• Definition 4: Structural equations