Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Probabilistic 3D Correspondence Prediction from Sparse Unsegmented Images

Krithika Iyer, Shireen Y. Elhabian

TL;DR

This work tackles the challenge of building statistical shape models from sparse, unsegmented medical images while providing reliable uncertainty quantification. It introduces SPI-CorrNet, a teacher–student framework that learns a shape prior from surface meshes and predicts $M$ 3D correspondences $\mathcal{C}_j^{I}$ directly from sparse image data, using a probabilistic encoder to model $p(\mathbf{z}^I)= \mathcal{N}(\mathbf{z}^I|\mu_{\mathbf{z}^I}, \operatorname{log}\sigma_{\mathbf{z}^I})$ and capture aleatoric uncertainty. Training uses three losses, $\mathcal{L}_{S}$, $\mathcal{L}_{EA}$, and $\mathcal{L}_{PR}$, to fuse the shape prior with image representations and refine predictions, while uncertainty estimation is integrated via sampling from the latent distribution. Empirically, on the Left Atrium LGE-MRI and AbdomenCT-1K liver datasets, SPI-CorrNet achieves competitive accuracy and well-calibrated uncertainty under sparse inputs, enabling robust, data-efficient, uncertainty-aware SSM for clinical decision support. Overall, the approach removes the need for ground-truth PDM supervision, improves efficiency for sparse data, and opens path toward alignment-free or alignment-robust probabilistic shape modeling in medical imaging.

Abstract

The study of physiology demonstrates that the form (shape)of anatomical structures dictates their functions, and analyzing the form of anatomies plays a crucial role in clinical research. Statistical shape modeling (SSM) is a widely used tool for quantitative analysis of forms of anatomies, aiding in characterizing and identifying differences within a population of subjects. Despite its utility, the conventional SSM construction pipeline is often complex and time-consuming. Additionally, reliance on linearity assumptions further limits the model from capturing clinically relevant variations. Recent advancements in deep learning solutions enable the direct inference of SSM from unsegmented medical images, streamlining the process and improving accessibility. However, the new methods of SSM from images do not adequately account for situations where the imaging data quality is poor or where only sparse information is available. Moreover, quantifying aleatoric uncertainty, which represents inherent data variability, is crucial in deploying deep learning for clinical tasks to ensure reliable model predictions and robust decision-making, especially in challenging imaging conditions. Therefore, we propose SPI-CorrNet, a unified model that predicts 3D correspondences from sparse imaging data. It leverages a teacher network to regularize feature learning and quantifies data-dependent aleatoric uncertainty by adapting the network to predict intrinsic input variances. Experiments on the LGE MRI left atrium dataset and Abdomen CT-1K liver datasets demonstrate that our technique enhances the accuracy and robustness of sparse image-driven SSM.

Probabilistic 3D Correspondence Prediction from Sparse Unsegmented Images

TL;DR

This work tackles the challenge of building statistical shape models from sparse, unsegmented medical images while providing reliable uncertainty quantification. It introduces SPI-CorrNet, a teacher–student framework that learns a shape prior from surface meshes and predicts 3D correspondences directly from sparse image data, using a probabilistic encoder to model and capture aleatoric uncertainty. Training uses three losses, , , and , to fuse the shape prior with image representations and refine predictions, while uncertainty estimation is integrated via sampling from the latent distribution. Empirically, on the Left Atrium LGE-MRI and AbdomenCT-1K liver datasets, SPI-CorrNet achieves competitive accuracy and well-calibrated uncertainty under sparse inputs, enabling robust, data-efficient, uncertainty-aware SSM for clinical decision support. Overall, the approach removes the need for ground-truth PDM supervision, improves efficiency for sparse data, and opens path toward alignment-free or alignment-robust probabilistic shape modeling in medical imaging.

Abstract

The study of physiology demonstrates that the form (shape)of anatomical structures dictates their functions, and analyzing the form of anatomies plays a crucial role in clinical research. Statistical shape modeling (SSM) is a widely used tool for quantitative analysis of forms of anatomies, aiding in characterizing and identifying differences within a population of subjects. Despite its utility, the conventional SSM construction pipeline is often complex and time-consuming. Additionally, reliance on linearity assumptions further limits the model from capturing clinically relevant variations. Recent advancements in deep learning solutions enable the direct inference of SSM from unsegmented medical images, streamlining the process and improving accessibility. However, the new methods of SSM from images do not adequately account for situations where the imaging data quality is poor or where only sparse information is available. Moreover, quantifying aleatoric uncertainty, which represents inherent data variability, is crucial in deploying deep learning for clinical tasks to ensure reliable model predictions and robust decision-making, especially in challenging imaging conditions. Therefore, we propose SPI-CorrNet, a unified model that predicts 3D correspondences from sparse imaging data. It leverages a teacher network to regularize feature learning and quantifies data-dependent aleatoric uncertainty by adapting the network to predict intrinsic input variances. Experiments on the LGE MRI left atrium dataset and Abdomen CT-1K liver datasets demonstrate that our technique enhances the accuracy and robustness of sparse image-driven SSM.
Paper Structure (17 sections, 3 equations, 4 figures, 1 table)

This paper contains 17 sections, 3 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Proposed Model
  5. Training
  6. Uncertainty Estimation:
  7. Dataset and Evaluation
  8. Datasets
  9. Metrics
  10. Results
  11. Conclusion
  12. Acknowledgements
  13. Appendix
  14. Dataset Details
  15. Hyperparamters
  16. ...and 2 more sections

Figures (4)

  • Figure 1: Architecture: (A) Base model SCorP iyer2024scorp with the teacher (surface autoencoder and IM-NET decoder) and student (image encoder) network. Proposed modifications for SPI-CorrNet: student network to handle (B) orthogonal image slices with a probabilistic image encoder and (C) full images with a probabilistic image encoder.
  • Figure 2: Performance metrics: Boxplots show performance metrics for held-out test samples from the LA and liver datasets. Compactness plots show cumulative variation captured by PCA modes. Comp = Compactness, Spec = Specificity, Gen = Generalization.
  • Figure 3: Uncertainty Calibration: Scatter plots and Pearson R coefficients show the correlation between estimated uncertainty and P2S error across test sets. Heatmaps on a representative mesh display average P2S error and aleatoric uncertainty, highlighting spatial correlation.
  • Figure 4: Ablation Experiments: (A) Box plots illustrate performance metrics for inliers, image outliers, and shape outliers, with example image slices. (B) Box plots compare performance metrics across varying slice thickness. (C) r-score for correlation between estimated uncertainty and P2S error for different slice thickness models. (D) Liver outlier detection results, with slices of outlier images and a plot showing total prediction uncertainty and P2S error, averaged across each shape. High-error, low-uncertainty outliers are highlighted in red.