Multi-modality fusion using canonical correlation analysis methods: Application in breast cancer survival prediction from histology and genomics

TL;DR

This work investigates the use of canonical correlation analysis (CCA) and penalized variants of CCA (pCCA), and proposes a two-stage prediction pipeline using pCCA embeddings generated with deflation for latent variable prediction by combining all the above.

Abstract

The availability of multi-modality datasets provides a unique opportunity to characterize the same object of interest using multiple viewpoints more comprehensively. In this work, we investigate the use of canonical correlation analysis (CCA) and penalized variants of CCA (pCCA) for the fusion of two modalities. We study a simple graphical model for the generation of two-modality data. We analytically show that, with known model parameters, posterior mean estimators that jointly use both modalities outperform arbitrary linear mixing of single modality posterior estimators in latent variable prediction. Penalized extensions of CCA (pCCA) that incorporate domain knowledge can discover correlations with high-dimensional, low-sample data, whereas traditional CCA is inapplicable. To facilitate the generation of multi-dimensional embeddings with pCCA, we propose two matrix deflation schemes that enforce desirable properties exhibited by CCA. We propose a two-stage prediction pipeline using pCCA embeddings generated with deflation for latent variable prediction by combining all the above. On simulated data, our proposed model drastically reduces the mean-squared error in latent variable prediction. When applied to publicly available histopathology data and RNA-sequencing data from The Cancer Genome Atlas (TCGA) breast cancer patients, our model can outperform principal components analysis (PCA) embeddings of the same dimension in survival prediction.

Figures (10)

• Figure 1: Multi-scale cancer data: Information about the same cancer captured at the organism level (clinical data), the organ level (radiology images), the tissue level (histopathology images) and gene expression levels (RNA-sequencing etc.) should be aggregated together effectively to characterize the underlying cancer.
• Figure 2: Overview: (a) Our proposed two-stage model for prediction takes in two modalities ($\mathbf{X}_i$, $\mathbf{Y}_i$) and utilizes pCCA with deflation to generate embeddings ($\tilde{\mathbf{X}}_i$, $\tilde{\mathbf{Y}}_i$) in an unsupervised manner. The embeddings are concatenated and fed to a (potentially) supervised prediction module for label prediction. (b) To generate embeddings with pCCA, we make use of an iterative deflation scheme where each iteration $j$ identifies canonical weights $\mathbf{u}_j, \mathbf{v}_j$. The final embeddings ($\tilde{\mathbf{X}}_i$, $\tilde{\mathbf{Y}}_i$) are generated by taking the product $\mathbf{U}^T \mathbf{X}_i$ and $\mathbf{V}^T \mathbf{Y}_i$ where $\mathbf{U} = \mathbf{U}_{1:K}= [\mathbf{u}_1 \dots \mathbf{u}_K]$ and $\mathbf{V} = \mathbf{V}_{1:K} = [\mathbf{v}_1 \dots \mathbf{v}_K]$.
• Figure 3: A graphical model for two-modalities bach2005probabilistic. The latent variable of interest ($\mathbf{z}$) influences the two observed modality variables $\mathbf{x}$ and $\mathbf{y}$ (Theorem \ref{['theorem:prob_cca']}). For the breast cancer survival prediction setting, $\mathbf{z}$ is the survival status, $\mathbf{x}$ the genomics feature and $\mathbf{y}$ are the imaging features.
• Figure 4: CCA, SCCA and GN-SCCA aim to identify the first set of canonical weights $\mathbf{u}_1$ and $\mathbf{v}_1$ such that the correlation between $\mathbf{u}^T \mathbf{X}$ and $\mathbf{v}^T \mathbf{Y}$ is maximized, subject to no constraints (CCA), sparsity constraints on $\mathbf{u}, \mathbf{v}$ (SCCA), and graph-based smoothness constraints on $\mathbf{u}, \mathbf{v}$ (GN-SCCA). This can be repeated to identify later sets $(\mathbf{u}_k, \mathbf{v}_k), k> 1$ using deflation schemes. Here, the first two sets of canonical weights $(\mathbf{u}_1, \mathbf{v}_1$) and $(\mathbf{u}_2, \mathbf{v}_2$) are shown.
• Figure 5: Illustrations of the three deflation schemes used

Theorems & Definitions (16)

• Theorem 1: CCA as ML estimators, Theorem 2 bach2005probabilistic
• Theorem 2
• Lemma 1
• Lemma 2
• Theorem A.1
• Lemma 3
• proof
• Lemma 4
• proof
• Corollary 1