From Fibers to Cells: Fourier-Based Registration Enables Virtual Cresyl Violet Staining From 3D Polarized Light Imaging

TL;DR

The paper tackles the challenge of linking cytoarchitecture with fiber architecture by converting 3D-PLI maps into virtual Cresyl violet stainings. It introduces a supervised image-to-image translation framework augmented with a Fourier-based online registration head to align training pairs despite residual misalignments, using reconstruction, style (Gram or GAN), and equivariance losses. Empirical results show that Gram-based style with online registration (Gram+Reg) provides the best balance between pixel fidelity and anatomical plausibility, enabling reliable localization of larger gray-matter cells and enabling downstream analyses like cell segmentation and laminar profiling. While not yet robust enough for full cytoarchitectonic mapping, the approach offers a scalable path toward joint cross-modal analysis and larger-scale synthetic staining across sections and brains, with potential extensions to additional stains and domain-specific encoders.

Abstract

Comprehensive assessment of the various aspects of the brain's microstructure requires the use of complementary imaging techniques. This includes measuring the spatial distribution of cell bodies (cytoarchitecture) and nerve fibers (myeloarchitecture). The gold standard for cytoarchitectonic analysis is light microscopic imaging of cell-body stained tissue sections. To reveal the 3D orientations of nerve fibers, 3D Polarized Light Imaging (3D-PLI) has been introduced, a method that is label-free and allows subsequent staining of sections after 3D-PLI measurement. By post-staining for cell bodies, a direct link between fiber- and cytoarchitecture can potentially be established in the same section. However, inevitable distortions introduced during the staining process make a costly nonlinear and cross-modal registration necessary in order to study the detailed relationships between cells and fibers in the images. In addition, the complexity of processing histological sections for post-staining only allows for a limited number of such samples. In this work, we take advantage of deep learning methods for image-to-image translation to generate a virtual staining of 3D-PLI that is spatially aligned at the cellular level. We use a supervised setting, building on a unique dataset of brain sections, to which Cresyl violet staining has been applied after 3D-PLI measurement. To ensure high correspondence between both modalities, we address the misalignment of training data using Fourier-based registration. In this way, registration can be efficiently calculated during training for local image patches of target and predicted staining. We demonstrate that the proposed method can predict a Cresyl violet staining from 3D-PLI, resulting in a virtual staining that exhibits plausible patterns of cell organization in gray matter, with larger cell bodies being localized at their expected positions.

Figures (11)

• Figure 1: The proposed virtual staining workflow. (A) Preprocessing of 3D-PLI sections that were post-stained with Cresyl violet. As paired training data, regions of interest (ROIs) are manually cropped (red boxes) and affine registered using large blood vessels as landmarks (green marker). Background pixels in train sections are masked, and retardation values are scaled using gamma correction for visualization purposes. (B) Training of a U-Net model using patches, extracted from same random locations (yellow boxes) in 3D-PLI modalities direction $\varphi$, retardation $\sin \delta$, transmittance $T_T$ and the Cresyl violet staining. 3D-PLI patches are used as input to the model to predict a virtual Cresyl violet staining. The Cresyl violet patch acts as target and is rigidly aligned with the prediction during the training procedure. The alignment is performed by our proposed online registration head using Fourier-based correlation of pixels. A loss $\mathcal{L}$ is computed between aligned target and prediction. (C) Inference using the trained U-Net model to virtually stain unseen sections or ROIs. Inputs are divided into overlapping tiles, which are processed independently by the U-Net model. The predictions are then stitched back together to form the complete virtual staining.
• Figure 2: Data modalities and registration challenges for training section 544. (A-C) 3D-PLI parameter maps: Transmittance, retardation and fiber orientation in HSV color space (hue: fiber direction; saturation/brightness: retardation). Background pixels are masked for visualization purposes only. (D) Affine registered Cresyl violet staining. The pial surface of the 3D-PLI acquisition is shown as a contour plot in D for reference. Between both data acquisitions remains a nonlinear misalignment that cannot be resolved by a global affine transformation. At a local scale, the remaining misalignment is approximately linear. Yellow arrows indicate blood vessels that can be used as mutual registration landmarks for coarse alignment.
• Figure 3: Illustration of the proposed virtual staining approach. Patches of 3D-PLI parameter maps transmittance $I_T$, retardation $\sin \delta$ (scaled using gamma correction for visualization), and direction $\varphi$ are used as input to a 2D convolutional U-Net model as generator to predict a virtual Cresyl violet staining. An online registration head estimates a rigid transformation $R$ between a coarsely aligned Cresyl violet target patch and the prediction via Fourier-based registration. Transformation $R$ is used to align target and prediction at the patch level. We calculate three distinct loss components: $\mathcal{L}_R$, $\mathcal{L}_S$ and $\mathcal{L}_E$. Reconstruction loss $\mathcal{L}_R$ performs a pixel-wise comparison between prediction and aligned target. Style loss $\mathcal{L}_S$ compares feature maps of a VGG network encoder using Gram matrices to mimic the style of the target image. Equivariance loss $\mathcal{L}_E$ applies the same U-Net model a second time to a rotated version of the input by rotation $\Omega$. The output is compared with the prediction rotated by same rotation $\Omega$, which promotes stability and avoids learning a constant shift of pixels in the prediction.
• Figure 4: Localization of train and test data. (A) Seven sections used for training (blue stripes) and one section used for testing (red stripe) were taken at the level of the central sulcus (CS; yellow dashed lines). Locations are shown on top of the 3D reconstructed blockface of the brain for reference. Train and test data are 0.6 mm apart from each other. (B) Selected locations of train and test regions of interest (ROIs), which are used for training and testing the models. The images show ROIs from each of the train and test sections on top of globally affine registered Cresyl violet images. Black contour plots outline the pial surface of corresponding 3D-PLI sections for reference.
• Figure 5: Overview of ROIs used for the evaluation, which represent distinct cellular architectures. They were extracted from section 559, located 0.6 mm apart from the training sections. Embedded windows show magnified details inside each ROI. Columns each show one of the four test ROIs taken from the anterior subdivision of the primary motor cortex (4a), the hippocampal cornu Ammonis (CA) region, temporal cortical area TE, and parts of the putamen and globus pallidus (GP; a: GP Pars interna; b: GP Pars externa) as subcortical nuclei. The first three rows demonstrate 3D-PLI modalities transmittance, retardation (scaled using gamma correction for visualization) and fiber orientation in HSV color space (hue: fiber direction; saturation/brightness: retardation). The 3D-PLI modalities are compared to the registered target Cresyl violet and predicted virtual staining.