3D-Guided Scalable Flow Matching for Generating Volumetric Tissue Spatial Transcriptomics from Serial Histology

TL;DR

HoloTea introduces a 3D-aware, generative framework for imputing volumetric tissue spatial transcriptomics from serial H&E sections by integrating adjacent-section context via a retrieval-conditioned flow-matching pipeline. The method combines adjacent-section conditioning with ControlNet-style injections, biology-aligned z-aware priors (a learned ZINB prior and a spatial-empirical prior), and a scalable Global Set Attention that enables near-linear growth with stack depth. Across public HER2 and ST-Data datasets and a large in-house embryo 3D stack, HoloTea achieves superior 3D expression fidelity and cross-slice coherence compared to 2D baselines and existing 3D methods, with ablations confirming the complementary value of priors, cross-section conditioning, and global-context mechanisms. The approach scales well to large stacks and holds promise for building accurate 3D virtual tissues, accelerating biomarker discovery and understanding of spatial biology in health and disease.

Abstract

A scalable and robust 3D tissue transcriptomics profile can enable a holistic understanding of tissue organization and provide deeper insights into human biology and disease. Most predictive algorithms that infer ST directly from histology treat each section independently and ignore 3D structure, while existing 3D-aware approaches are not generative and do not scale well. We present Holographic Tissue Expression Inpainting and Analysis (HoloTea), a 3D-aware flow-matching framework that imputes spot-level gene expression from H&E while explicitly using information from adjacent sections. Our key idea is to retrieve morphologically corresponding spots on neighboring slides in a shared feature space and fuse this cross section context into a lightweight ControlNet, allowing conditioning to follow anatomical continuity. To better capture the count nature of the data, we introduce a 3D-consistent prior for flow matching that combines a learned zero-inflated negative binomial (ZINB) prior with a spatial-empirical prior constructed from neighboring sections. A global attention block introduces 3D H&E scaling linearly with the number of spots in the slide, enabling training and inference on large 3D ST datasets. Across three spatial transcriptomics datasets spanning different tissue types and resolutions, HoloTea consistently improves 3D expression accuracy and generalization compared to 2D and 3D baselines. We envision HoloTea advancing the creation of accurate 3D virtual tissues, ultimately accelerating biomarker discovery and deepening our understanding of disease.

Figures (7)

• Figure 1: HoloTea leverages spatial neighborhood information from adjacent sections to improve 3D consistency in H&E to ST imputation.
• Figure 2: Method overview. For each query spot on section $z$, HoloTea retrieves nearest neighbors from sections $z{\pm}1$ using feature-space similarity, aggregates them into an adjacent token, and uses a ControlNet to condition the flow-matching denoiser with ZINB-based priors.
• Figure 3: Comparison of spatial gene-expression predictions for AHSP: our method (top), STFlow (bottom), and ground truth (right) across different flow steps.
• Figure S1: Neighbor consistency of joint clustering across 26 unseen sections. GT: ground truth, Imp: imputed, std: standard deviation.
• Figure S2: Joint clustering of the ground truth expression across all 26 sections not seen during training.