ASIGN: An Anatomy-aware Spatial Imputation Graphic Network for 3D Spatial Transcriptomics

TL;DR

This work tackles the high cost and limited 3D context of spatial transcriptomics by proposing ASIGN, a framework that imputes 3D ST data from 3D WSI sections and a single 2D ST slide. It introduces a 3D graph construction pipeline driven by cross-layer registration, a Multi-Level Spatial Attention Graph Network (MASGNet) for integrating multi-resolution and 3D information, and a Cross-layer Imputation (CLI) module to propagate known labels across layers. The method achieves state-of-the-art performance across three public datasets, with strong cross-sample generalization and improved accuracy for clinically relevant markers like ERBB2 and MDK. By combining anatomy-aware 3D structure with cost-efficient data sources, ASIGN demonstrates practical potential for accelerating 3D ST analyses in clinical settings.

Abstract

Spatial transcriptomics (ST) is an emerging technology that enables medical computer vision scientists to automatically interpret the molecular profiles underlying morphological features. Currently, however, most deep learning-based ST analyses are limited to two-dimensional (2D) sections, which can introduce diagnostic errors due to the heterogeneity of pathological tissues across 3D sections. Expanding ST to three-dimensional (3D) volumes is challenging due to the prohibitive costs; a 2D ST acquisition already costs over 50 times more than whole slide imaging (WSI), and a full 3D volume with 10 sections can be an order of magnitude more expensive. To reduce costs, scientists have attempted to predict ST data directly from WSI without performing actual ST acquisition. However, these methods typically yield unsatisfying results. To address this, we introduce a novel problem setting: 3D ST imputation using 3D WSI histology sections combined with a single 2D ST slide. To do so, we present the Anatomy-aware Spatial Imputation Graph Network (ASIGN) for more precise, yet affordable, 3D ST modeling. The ASIGN architecture extends existing 2D spatial relationships into 3D by leveraging cross-layer overlap and similarity-based expansion. Moreover, a multi-level spatial attention graph network integrates features comprehensively across different data sources. We evaluated ASIGN on three public spatial transcriptomics datasets, with experimental results demonstrating that ASIGN achieves state-of-the-art performance on both 2D and 3D scenarios. Code is available at https://github.com/hrlblab/ASIGN.

Figures (8)

• Figure 1: ASIGN presents a novel problem setting for 3D spatial transcriptomic (ST). Unlike recent methods that focus on either (a) directly predicting 2D ST from 2D WSI or (b) performing a full 3D WSI to 3D ST prediction, our proposed ASIGN method generates a 3D ST volume by combining 3D WSI histology sections with a single 2D ST slide. This approach provides a balanced solution—more precise than "free but less accurate" 3D predictions, yet far more affordable than the "precise but prohibitively expensive" acquisition of full 3D ST for every slide.
• Figure 2: Overall framework of our proposed ASIGN approach. ASIGN begins with a global alignment process to estimate overlap and similarity across layers, constructing 3D graph connections. Then, MSAGNet, integrated within ASIGN, enhances information fusion across layers, neighboring regions, and resolutions for comprehensive feature integration. Finally, a CLI block propagates predictions from partially known labels, which are weighted with MSAGNet’s model predictions to generate the final output.
• Figure 3: Architecture of MSAGNet and the CLI block. MSAGNet comprises cross-attention layers, GAT blocks, and Transformer layers to integrate and aggregate features across multiple resolution levels, 3D sample levels, and patch-self levels, respectively. The CLI block employs a label propagation strategy to diffuse known labels to unknown patches. A self-adaptive weighting mechanism merges the final results from both MSAGNet and the CLI block.
• Figure 4: Quantitative comparison between different methods. This figure presents the box plot of different methods with Pearson correlation coefficient (PCC), mean squared error (MSE), and mean absolute error (MAE). Welch’s t-test is used to assess the statistical significance of differences between other method and ASIGN. P-values below 0.05 are considered significant and are annotated with $\ast$.
• Figure 5: Qualitative comparison between different methods. This figure presents the ST imputation performance for predicting the distribution of cancer markers, specifically (A) ERBB2 and (B) MDK, within the WSIs. The MSE between the ground truth and the predicted values is presented for each model.