Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Extending 2D foundational DINOv3 representations to 3D segmentation of neonatal brain MR images

Annayah Usman, Behraj Khan, Tahir Qasim Syed

TL;DR

The method demonstrates that volumetric anatomical structure could be recovered from frozen 2D foundation representations through structured compositional decoding, and offers a principled and generalizable extension for foundation models for 3D medical applications.

Abstract

Precise volumetric delineation of hippocampal structures is essential for quantifying neurodevelopmental trajectories in pre-term and term infants, where subtle morphological variations may carry prognostic significance. While foundation encoders trained on large-scale visual data offer discriminative representations, their 2D formulation is a limitation with respect to the $3$D organization of brain anatomy. We propose a volumetric segmentation strategy that reconciles this tension through a structured window-based disassembly-reassembly mechanism: the global MRI volume is decomposed into non-overlapping 3D windows or sub-cubes, each processed via a separate decoding arm built upon frozen high-fidelity features, and subsequently reassembled prior to a ground-truth correspendence using a dense-prediction head. This architecture preserves constant a decoder memory footprint while forcing predictions to lie within an anatomically consistent geometry. Evaluated on the ALBERT dataset for hippocampal segmentation, the proposed approach achieves a Dice score of 0.65 for a single 3D window. The method demonstrates that volumetric anatomical structure could be recovered from frozen 2D foundation representations through structured compositional decoding, and offers a principled and generalizable extension for foundation models for 3D medical applications.

Extending 2D foundational DINOv3 representations to 3D segmentation of neonatal brain MR images

TL;DR

The method demonstrates that volumetric anatomical structure could be recovered from frozen 2D foundation representations through structured compositional decoding, and offers a principled and generalizable extension for foundation models for 3D medical applications.

Abstract

Precise volumetric delineation of hippocampal structures is essential for quantifying neurodevelopmental trajectories in pre-term and term infants, where subtle morphological variations may carry prognostic significance. While foundation encoders trained on large-scale visual data offer discriminative representations, their 2D formulation is a limitation with respect to the D organization of brain anatomy. We propose a volumetric segmentation strategy that reconciles this tension through a structured window-based disassembly-reassembly mechanism: the global MRI volume is decomposed into non-overlapping 3D windows or sub-cubes, each processed via a separate decoding arm built upon frozen high-fidelity features, and subsequently reassembled prior to a ground-truth correspendence using a dense-prediction head. This architecture preserves constant a decoder memory footprint while forcing predictions to lie within an anatomically consistent geometry. Evaluated on the ALBERT dataset for hippocampal segmentation, the proposed approach achieves a Dice score of 0.65 for a single 3D window. The method demonstrates that volumetric anatomical structure could be recovered from frozen 2D foundation representations through structured compositional decoding, and offers a principled and generalizable extension for foundation models for 3D medical applications.
Paper Structure (7 sections, 2 equations, 2 figures, 2 tables)

This paper contains 7 sections, 2 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Experiments and Results
  5. Experiments
  6. Results
  7. Conclusion

Figures (2)

  • Figure 1: (a) A 3D MRI volume is decomposed into sub-cubes, encoded using a frozen 2D DINOv3 ViT with depth embedding, decoded via a lightweight head, and reassembled for global supervision. (b) Two-pass training: detached global loss computation followed by gradient-correct sub-cube backpropagation.
  • Figure 2: Qualitative comparison of hippocampus segmentation. The single-cube model shows improved anatomical coherence and boundary accuracy compared to the multi-cube setting.