Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integrating Genomics into Multimodal EHR Foundation Models

Jonathan Amar, Edward Liu, Alessandra Breschi, Liangliang Zhang, Pouya Kheradpour, Sylvia Li, Lisa Soleymani Lehmann, Alessandro Giulianelli, Matt Edwards, Yugang Jia, David Nola, Raghav Mani, Pankaj Vats, Jesse Tetreault, T. J. Chen, Cory Y. McLean

TL;DR

The paper presents a multimodal EHR foundation model that integrates 3481 polygenic risk scores (PRS) with traditional EHR data, trained end-to-end on the All of Us cohort to learn rich, cross-modal health trajectories. By employing cross-attention or adapter-based embeddings, the model jointly reasons over static genomic features and dynamic clinical events, yielding improved discrimination for diseases such as Type 2 Diabetes ($AUROC$ improved by $+0.025$; $AUPRC$ by $+0.041$) and enabling novel risk-scoring via path-computing probabilities. Across analyses, PRS integration shows modest yet significant gains in disease prediction and demonstrates alignment with known PRS signals, with transfer-learning experiments indicating efficient adaptation to downstream tasks like stroke and COPD. The approach advances personalized, equitable real-world evidence by enabling richer health representations, dynamic risk assessment, and potential digital-twin-like simulations, while acknowledging biases, PRS limitations, and calibration needs for clinical deployment.

Abstract

This paper introduces an innovative Electronic Health Record (EHR) foundation model that integrates Polygenic Risk Scores (PRS) as a foundational data modality, moving beyond traditional EHR-only approaches to build more holistic health profiles. Leveraging the extensive and diverse data from the All of Us (AoU) Research Program, this multimodal framework aims to learn complex relationships between clinical data and genetic predispositions. The methodology extends advancements in generative AI to the EHR foundation model space, enhancing predictive capabilities and interpretability. Evaluation on AoU data demonstrates the model's predictive value for the onset of various conditions, particularly Type 2 Diabetes (T2D), and illustrates the interplay between PRS and EHR data. The work also explores transfer learning for custom classification tasks, showcasing the architecture's versatility and efficiency. This approach is pivotal for unlocking new insights into disease prediction, proactive health management, risk stratification, and personalized treatment strategies, laying the groundwork for more personalized, equitable, and actionable real-world evidence generation in healthcare.

Integrating Genomics into Multimodal EHR Foundation Models

TL;DR

The paper presents a multimodal EHR foundation model that integrates 3481 polygenic risk scores (PRS) with traditional EHR data, trained end-to-end on the All of Us cohort to learn rich, cross-modal health trajectories. By employing cross-attention or adapter-based embeddings, the model jointly reasons over static genomic features and dynamic clinical events, yielding improved discrimination for diseases such as Type 2 Diabetes ( improved by ; by ) and enabling novel risk-scoring via path-computing probabilities. Across analyses, PRS integration shows modest yet significant gains in disease prediction and demonstrates alignment with known PRS signals, with transfer-learning experiments indicating efficient adaptation to downstream tasks like stroke and COPD. The approach advances personalized, equitable real-world evidence by enabling richer health representations, dynamic risk assessment, and potential digital-twin-like simulations, while acknowledging biases, PRS limitations, and calibration needs for clinical deployment.

Abstract

This paper introduces an innovative Electronic Health Record (EHR) foundation model that integrates Polygenic Risk Scores (PRS) as a foundational data modality, moving beyond traditional EHR-only approaches to build more holistic health profiles. Leveraging the extensive and diverse data from the All of Us (AoU) Research Program, this multimodal framework aims to learn complex relationships between clinical data and genetic predispositions. The methodology extends advancements in generative AI to the EHR foundation model space, enhancing predictive capabilities and interpretability. Evaluation on AoU data demonstrates the model's predictive value for the onset of various conditions, particularly Type 2 Diabetes (T2D), and illustrates the interplay between PRS and EHR data. The work also explores transfer learning for custom classification tasks, showcasing the architecture's versatility and efficiency. This approach is pivotal for unlocking new insights into disease prediction, proactive health management, risk stratification, and personalized treatment strategies, laying the groundwork for more personalized, equitable, and actionable real-world evidence generation in healthcare.

Paper Structure

This paper contains 39 sections, 2 equations, 9 figures, 5 tables.

Table of Contents

  1. Introduction
  2. General introduction and key idea
  3. Overview of Foundational Models (FM) in Electronic Health Records (EHR)
  4. Using PRS for various risk assessments
  5. Contributions and data
  6. Methods
  7. Data
  8. Data pre-processing
  9. Tokenization details
  10. Genomics to PRS
  11. Models and multimodality
  12. Multimodal static features
  13. Naming
  14. Note on dynamic modalities
  15. Positional embedding and training label note
  16. ...and 24 more sections

Figures (9)

  • Figure 1: Summary of the Electronic Health Records (EHR) foundational model with multi-modality genomics using polygenic risk scores (PRS). (Top) Model is trained using generic next token prediction on the All of Us dataset. (Bottom) The FM is evaluated on held-out data for various conditions - Type 2 Diabetes (T2D), Schizophrenia (SCZ), depression (DEPR), and Coronary Artery Disease (CAD) - and medical history settings (increasing EHR made available). AUCs compare the multi-modal EHR+PRS (GPT-PRS-CROSS) against the EHR (GPT-EHR) only foundation model.
  • Figure 2: Proposed framework for integrating various data modalities, such as polygenic risk scores (PRS) into Electronic Health Records (EHR) foundational models. (top) Architecture suggested: details how different inputs are processed, embedded, and then fed into Transformer architectures, including variations with inserted prefix/dynamic soft tokens and cross-attention mechanisms, to generate health trajectories (M') for downstream tasks. This work focuses on comparing the basic EHR only model against the transformer with cross-attention for PRS. (bottom) Data calculation pipeline from raw genomics GWAS to PRS on All of Us dataset.
  • Figure 3: AUC Comparison between GPT-EHR and GPT-PRS-CROSS, on 10 year prediction with demographic tokens only, for various conditions: type 2 diabetes (T2D), schizophrenia (SCZ), depression (DEPR), and coronary artery disease (CAD). Positive number of cases for each task is shown. Test size = 12,009. T2D has non-overlapping bootstrapped confidence intervals. The multimodal model consistently improves over the EHR only baseline.
  • Figure 4: Precision-Recall Curve difference between model EHR+PRS (B) and model with EHR only (A) for 10-year T2D prediction, using demographic tokens.
  • Figure 5: Meta-analysis of $\Delta$AUROC across tasks
  • ...and 4 more figures