An effective interactive brain cytoarchitectonic parcellation framework using pretrained foundation model

TL;DR

This work tackles scalable cytoarchitectonic brain parcellation under limited expert labels and across staining modalities by lever- aging a frozen DINOv3 vision transformer as a feature extractor. A lightweight decoder, informed by sparse scribbles and enhanced through multi-layer feature fusion, enables rapid, interactive segmentation with real-time feedback. The approach uses overlapped patch inference and a Napari-based GUI, and a composite loss to handle class imbalance, achieving substantial improvements over nnU-Net on macaque V1 laminar segmentation and demonstrating cross-stain anatomical correspondence. Overall, foundation-model-driven interactive segmentation offers a scalable, practical pathway for large-scale cytoarchitectonic mapping across datasets and species.

Abstract

Cytoarchitectonic mapping provides anatomically grounded parcellations of brain structure and forms a foundation for integrative, multi-modal neuroscience analyses. These parcellations are defined based on the shape, density, and spatial arrangement of neuronal cell bodies observed in histological imaging. Recent works have demonstrated the potential of using deep learning models toward fully automatic segmentation of cytoarchitectonic areas in large-scale datasets, but performance is mainly constrained by the scarcity of training labels and the variability of staining and imaging conditions. To address these challenges, we propose an interactive cytoarchitectonic parcellation framework that leverages the strong transferability of the DINOv3 vision transformer. Our framework combines (i) multi-layer DINOv3 feature fusion, (ii) a lightweight segmentation decoder, and (iii) real-time user-guided training from sparse scribbles. This design enables rapid human-in-the-loop refinement while maintaining high segmentation accuracy. Compared with training an nnU-Net from scratch, transfer learning with DINOv3 yields markedly improved performance. We also show that features extracted by DINOv3 exhibit clear anatomical correspondence and demonstrate the method's practical utility for brain region segmentation using sparse labels. These results highlight the potential of foundation-model-driven interactive segmentation for scalable and efficient cytoarchitectonic mapping.

Figures (5)

• Figure 1: Overview of the proposed interactive cytoarchitectonic parcellation framework. A pretrained DINOv3-B vision transformer serves as a frozen encoder. Multi-layer features from different depths are upsampled, aligned, and concatenated, then passed to a lightweight MLP head to produce the final segmentation mask. Sparse user-provided scribbles are used to fine-tune the decoder, enabling rapid, interactive parcellation.
• Figure 2: Interactive segmentation, in which segmentation accuracy is progressively improved through iterative refinement of user annotations.
• Figure 3: Predicted V1 cortical laminar segmentation on a test section. The method produces accurate and smooth layer boundaries in vertical regions where laminar structure is clearly preserved.
• Figure 4: Feature visualization and interactive segmentation results across three datasets. (a–c) Dataset 1 (Nissl-stained VISoR mouse brain), (d–f) Dataset 2 (DAPI-stained whole-slide images), and (g–i) Dataset 3 (thionin-stained sagittal sections). PCA projection of DINOv3 embeddings yields RGB maps with sharp anatomical boundaries. Scribble-based interactive segmentation results are shown for representative regions including Hippocampal formation (HIP), Facial Motor Nucleus (VII), Facial Nerve (VIIn), and Visual Cortex (VIS). For nucleus-stained data (Dataset 2), VIIn replaces VII due to reduced visibility of neuronal somata. Cytoarchitectonically similar structures are grouped when distinctions are subtle even for human experts (e.g., stratum oriens vs. stratum radiatum in HIP).
• Figure 5: High-resolution ($1~\mu$m) DINOv3 feature visualizations and interactive segmentation. Hippocampal regions from the three datasets are processed at higher input resolution, revealing finer laminar and subfield distinctions. DINOv3 embeddings become more discriminative at $1~\mu\mathrm{m}$ resolution, enabling accurate segmentation of thin structures such as the pyramidal layer and improved separation of subtle subregions including stratum lacunosum-moleculare and the dentate molecular layer.