EVA: Towards a universal model of the immune system

TL;DR

EVA introduces a novel cross-species, multimodal foundation model for immunology and inflammation that unifies transcriptomics across human and mouse with histology data to produce patient-level representations. By jointly training a transcriptomics encoder (EVA-RNA) and a histology encoder (EVA-H) and fusing them with a multimodal transformer, EVA demonstrates strong, transferable performance across a 39-task benchmark spanning discovery, preclinical translation, and clinical prediction, with clear scaling laws up to 300M RNA parameters. Mechanistic interpretability via sparse autoencoders reveals interpretable, cross-species concepts that map to conserved immune programs and tissue differentiation. EVA’s zero-shot perturbation, cross-species transfer, and end-to-end translational evaluations address key translational barriers while providing an open-access EVA-RNA release to accelerate community research in immune-mediated diseases. The work highlights the value of modality- and species-aware pretraining for translational biology and sets a benchmarking paradigm aligned with drug development priorities.

Abstract

The effective application of foundation models to translational research in immune-mediated diseases requires multimodal patient-level representations that can capture complex phenotypes emerging from multicellular interactions. Yet most current biological foundation models focus only on single-cell resolution and are evaluated on technical metrics often disconnected from actual drug development tasks and challenges. Here, we introduce EVA, the first cross-species, multimodal foundation model of immunology and inflammation, a therapeutic area where shared pathogenic mechanisms create unique opportunities for transfer learning. EVA harmonizes transcriptomics data across species, platforms, and resolutions, and integrates histology data to produce rich, unified patient representations. We establish clear scaling laws, demonstrating that increasing model size and compute translates to improvements in both pretraining and downstream tasks performance. We introduce a comprehensive evaluation suite of 39 tasks spanning the drug development pipeline: zero-shot target efficacy and gene function prediction for discovery, cross-species or cross-diseases molecular perturbations for preclinical development, and patient stratification with treatment response prediction or disease activity prediction for clinical trials applications. We benchmark EVA against several state-of-the-art biological foundation models and baselines on these tasks, and demonstrate state-of-the-art results on each task category. Using mechanistic interpretability, we further identify biological meaningful features, revealing intertwined representations across species and technologies. We release an open version of EVA for transcriptomics to accelerate research on immune-mediated diseases.

Figures (9)

• Figure 1: The EVA model architecture. (a) EVA-RNA pretraining with stochastic masked gene expression prediction and CLS token compression. (b) EVA multimodal architecture integrating gene embeddings from EVA-RNA with tile embeddings from EVA-H via a joint transformer. (c) EVA multimodal contrastive pretraining with multiple views per sample and InfoNCE objective. (d) UMAP of the training datasets embedded through EVA showing co-embedding of samples across species, technologies, and modalities. Interestingly, different modalities are embedded separately, a phenomenon observed and documented in other multimodal approaches like CLIP liang2022mind.
• Figure 2: Zero-shot drug efficacy predictions, for each disease. The number of stars indicates the number of targets perturbed for this drug. Each box plot represents the distribution of predicted efficacy over the whole cohort. Drugs are ranked by median predicted efficacy. Blue bar plots represent drugs with confirmed positive trial results, red bar plots represent drugs with negative results or no expected efficacy. $n$ stands for the number of patients in each cohort. Detailed methodology is reported in Section \ref{['sec:zsp']}.
• Figure 3: Cross-technologies and cross-species alignment at multiple levels in EVA-RNA. (a) Evolution of nearest neighbor median rank between orthologs based on their input embeddings. Immune genes are faster and better aligned than other groups. See Section \ref{['apx:nn-rank-evolution']} for methodology details. (b) UMAP of contextualized gene embeddings from layer 30 (N-1) at 5000 training steps and 550,000 training steps. The method is described in Section \ref{['apx:context-gene-emb']}. (c) UMAP of concept vectors extracted from the last CLS token of EVA-RNA with TopK sparse auto-encoder. Each point is a concept; the colors and markers correspond to the technologies and species among the 200 samples with the highest concept activation (prototypes). In boxes, we provide examples of 9 concepts, their interpretations, and the distribution of technologies and species across the 200 samples with the highest concept activations. The method is described in Section \ref{['method:sae']}.
• Figure 4: EVA-RNA exhibits predictable scaling behavior across pretraining and downstream evaluation. (a) Validation loss as a function of compute for five model sizes (7M-300M parameters). Loss follows a power-law relationship with no evidence of plateau. (b) Downstream task performance as a function of training steps across six evaluation categories, showing a clear improvement with continued pretraining. (c) PCA of sample embeddings at layers 29 (top) and 31 (bottom), colored by data source. Layer 29 (N-2) retains multi-dimensional structure, while layer 31 (N) collapses onto the first principal component, reflecting compression toward the sample-level reconstruction objective. (d) TwoNN intrinsic dimension across transformer layers at different training checkpoints. Early layers maintain high-dimensional representations throughout training, while later layers progressively compress the contextualized gene representations. This compression effect intensifies with training, with final layer showing increasingly sharp rank reduction at later checkpoints. See section \ref{['apx:intrisic-dimensionality']} for more details.
• Figure 5: WSI Preprocessing: From initial tissue segmentation to the extraction of localized tiles for model input.