Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-context principal component analysis

Kexin Wang, Salil Bhate, João M. Pereira, Joe Kileel, Matylda Figlerowicz, Anna Seigal

TL;DR

MCPCA extends PCA to multi-context data by modeling context-specific covariances as $\Sigma_i = A B_i A^T$ with a shared MCPC basis $A$ and context loadings $B_i$, forming a covariance tensor $T$ whose rank-$r$ CP-like decomposition yields MCPCs and their context weights. The authors develop a scalable MSPM-based algorithm, with a nonnegativity constraint on $B$, and demonstrate superior recovery, stability, and scalability compared to existing tensor methods. Across cancer genomics, single-cell profiling, perturb-seq benchmarking, phylogenetics, and literary-text embeddings, MCPCA uncovers axes of variation shared across subsets of contexts that PCA on pooled data or per-context analyses miss, including survival-linked cancer subgroups and cross-temporal debates in language. The framework provides interpretable, context-aware factors and context loadings, enabling downstream analyses such as survival prediction, functional enrichment, and evolutionary interpretation, with theoretical guarantees on identifiability and convergence under finite-sample noise.

Abstract

Principal component analysis (PCA) is a tool to capture factors that explain variation in data. Across domains, data are now collected across multiple contexts (for example, individuals with different diseases, cells of different types, or words across texts). While the factors explaining variation in data are undoubtedly shared across subsets of contexts, no tools currently exist to systematically recover such factors. We develop multi-context principal component analysis (MCPCA), a theoretical and algorithmic framework that decomposes data into factors shared across subsets of contexts. Applied to gene expression, MCPCA reveals axes of variation shared across subsets of cancer types and an axis whose variability in tumor cells, but not mean, is associated with lung cancer progression. Applied to contextualized word embeddings from language models, MCPCA maps stages of a debate on human nature, revealing a discussion between science and fiction over decades. These axes are not found by combining data across contexts or by restricting to individual contexts. MCPCA is a principled generalization of PCA to address the challenge of understanding factors underlying data across contexts.

Multi-context principal component analysis

TL;DR

MCPCA extends PCA to multi-context data by modeling context-specific covariances as with a shared MCPC basis and context loadings , forming a covariance tensor whose rank- CP-like decomposition yields MCPCs and their context weights. The authors develop a scalable MSPM-based algorithm, with a nonnegativity constraint on , and demonstrate superior recovery, stability, and scalability compared to existing tensor methods. Across cancer genomics, single-cell profiling, perturb-seq benchmarking, phylogenetics, and literary-text embeddings, MCPCA uncovers axes of variation shared across subsets of contexts that PCA on pooled data or per-context analyses miss, including survival-linked cancer subgroups and cross-temporal debates in language. The framework provides interpretable, context-aware factors and context loadings, enabling downstream analyses such as survival prediction, functional enrichment, and evolutionary interpretation, with theoretical guarantees on identifiability and convergence under finite-sample noise.

Abstract

Principal component analysis (PCA) is a tool to capture factors that explain variation in data. Across domains, data are now collected across multiple contexts (for example, individuals with different diseases, cells of different types, or words across texts). While the factors explaining variation in data are undoubtedly shared across subsets of contexts, no tools currently exist to systematically recover such factors. We develop multi-context principal component analysis (MCPCA), a theoretical and algorithmic framework that decomposes data into factors shared across subsets of contexts. Applied to gene expression, MCPCA reveals axes of variation shared across subsets of cancer types and an axis whose variability in tumor cells, but not mean, is associated with lung cancer progression. Applied to contextualized word embeddings from language models, MCPCA maps stages of a debate on human nature, revealing a discussion between science and fiction over decades. These axes are not found by combining data across contexts or by restricting to individual contexts. MCPCA is a principled generalization of PCA to address the challenge of understanding factors underlying data across contexts.
Paper Structure (62 sections, 18 theorems, 89 equations, 9 figures, 1 algorithm)

This paper contains 62 sections, 18 theorems, 89 equations, 9 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Results
  3. A model for factors shared across subsets of contexts
  4. Tensor decomposition estimates multi-context principal components
  5. An accurate and scalable algorithm for estimating MCPCs
  6. MCPCs of cancer
  7. Variation in single-cell gene expression heterogeneity across patients
  8. Benchmarking context-level representations for gene expression from MCPCA
  9. Examining intellectual history across time periods and genres of textual production
  10. Discussion
  11. Methods
  12. Tensor decomposition
  13. Choosing the rank
  14. Synthetic analysis
  15. Semi-synthetic analysis
  16. ...and 47 more sections

Key Result

Proposition SI-1.2

The MCPCA model $\Sigma_i = AB_i A^\mathsf{T}$ for all $i = 1, \ldots, k$ is equivalent to the tensor decomposition where $\mathbf{a}_j$ is the $j$th column of $A \in \mathbb{R}^{p \times r}$, and $\mathbf{b}_j$ is the $j$th column of $B \in \mathbb{R}^{k \times r}$.

Figures (9)

  • Figure 1: A model for multi-context principal components. A. (1) Data assigned to known contexts (here, blue and orange) plotted along the top two overall PCs, (2) and (3) Data in individual contexts, plotted in the space of overall PCs, the within-context PCs marked in blue and orange, respectively, and MCPCs shown in black. The MCPCs are non-orthogonal directions that capture axes of variation shared across contexts. B. (1) the parameters in MCPC: the matrix of MCPCs (reds) and the context loading matrix (blues). This example has five variables, four contexts, and three MCPCs. (2) The MCPCs are components of covariance. Each is weighted by the context loading to approximate the covariance matrices in each context. C. The covariance tensor is obtained by stacking the covariance matrices from each context.
  • Figure 2: ⁠Benchmarking MCPCA with synthetic and semi-synthetic data. A. Synthetic data generated from a standard multivariate Gaussian by transforming by a context-dependent diagonal scaling followed by a shared linear transformation. MCPCs marked as arrows in the transformed data. B. Runtime (in seconds) against accuracy (Ascore) for the 13 methods in the legend. MCPCA lower left, the fastest accurate method. C. Accuracy compared to sample size for the 13 methods listed in the legend. D. Semi-synthetic data schematic: data generated via background images of clouds and grass with digits superimposed. Ground truth context loadings shown in $3 \times 5$ array. E. Three MCPCs plotted via pixel intensities. Context loading matrix as heatmap. Digit 0 has high loading in context 2, digit 1 has high loading in context 3 and digit two has non-zero loadings in contexts 2 and 3. This matches ground truth in D. F. The four ground truth averaged digits (zero in context 2, two in context 2, one in context 3, two in context 3). G. Digits recovered via FFDIAG, NLS, PCA-STACK, QRJ1D, HO GSVD.
  • Figure 3: Multi-context principal components of cancer across tumors and patients. A. Gene-gene covariance tensor across cancer types. B. Context loadings from MCPC with rank 30. Colors indicate grouping into broad, multicancer and organ-specific MCPCs C. Distribution of scores of MCPC10 in patients with cancers of highest context loading. D. Scores of overall PCs 2 and 5 (those with highest similarity to MCPC10) in thyroid cancer. Color indicates MCPC10 score. Black line is projection of MCPC10. E. Scores of overall PCs 2 and 5 in pancreatic cancer. Color indicates MCPC10 score. Black line indicates projection of MCPC10 on PCs 2 and 5. Dashed circle indicates patients with highest score along MCPC10. F. Kaplan-Meier plot of patients with high and low MCPC10. $**$ indicates Cox Proportional-Hazards regression p-value $= 0.001$. G. Schematic of gene-gene covariance tensor across lung cancer patients. H. Distribution of context loadings of MCPC5 per patient grouped by cancer stage. I. Distribution of per-cell MCPC5 score in Stage I and IV patients; t-test p-value for difference in std of scores $=0.01$, p-value for difference in mean not significant. J. -Log10pvalues of association with cancer stage (maximum p-value for t-test difference between groups I/II vs III/IV and Spearman correlation with stage) for std of overall PCs 1-400. Red line indicates significance obtained by MCPC5.
  • Figure 4: ⁠Multi-context principal components of semantics across literary forms and time periods. A. Pipeline for feature extraction from contextualized embeddings of the word “human” in the Gutenberg dataset. B. Input to MCPC. Covariance matrices of human embeddings across literary forms (science and fiction) and time intervals (20 year intervals between 1800 and 1900). C. Context loadings of MCPCs across contexts. D. Graph summarizing temporal ordering of factors observed in both literary forms by their context loadings. E. Correlation of context variances ($y$ axis) and overlap between top 200 sentences ($x$ axis) with factors obtained by PCA in individual contexts and overall PCs for MCPC4. F. Correlation of context variances ($y$ axis) and overlap between top 200 sentences ($x$ axis) with factors obtained by PCA in individual contexts and overall PCs for MCPC6. G. Distributions of median scores of MCPC6 across works grouped by time and form. H. Annotation of MCPC6 using sentences with highest and lowest scores. I. Annotation of MCPC4 using sentences with highest and lowest scores.
  • Figure SI-1: A tensor decomposition of the covariance tensor. Illustrated for five variables measured across four contexts.
  • ...and 4 more figures

Theorems & Definitions (44)

  • Definition SI-1.1
  • Proposition SI-1.2
  • proof
  • Definition SI-2.1: Identifiability of latent variable models, see allman2009identifiability
  • Proposition SI-2.2
  • proof
  • Proposition SI-2.3
  • proof
  • Proposition SI-2.4
  • proof
  • ...and 34 more