Self-Supervised Representation Learning for Nerve Fiber Distribution Patterns in 3D-PLI

TL;DR

This paper tackles the challenge of obtaining observer-independent descriptors of nerve fiber architecture in high-resolution 3D-PLI data. It introduces 3D-Context Contrastive Learning (CL-3D), a self-supervised framework that samples positive pairs across nearby brain sections and uses targeted augmentations to learn robust texture embeddings from 3D-PLI parameter maps. CL-3D features capture fundamental fiber configurations, are robust to histological processing variations, and enable clustering, linear classification with minimal labels, and retrieval of specific fiber patterns such as U-fibers. The approach demonstrates strong descriptive power, alignment with cortical morphology, and potential for scalable brain mapping and atlas integration, including future extension to whole-brain datasets and cross-species analyses.

Abstract

A comprehensive understanding of the organizational principles in the human brain requires, among other factors, well-quantifiable descriptors of nerve fiber architecture. Three-dimensional polarized light imaging (3D-PLI) is a microscopic imaging technique that enables insights into the fine-grained organization of myelinated nerve fibers with high resolution. Descriptors characterizing the fiber architecture observed in 3D-PLI would enable downstream analysis tasks such as multimodal correlation studies, clustering, and mapping. However, best practices for observer-independent characterization of fiber architecture in 3D-PLI are not yet available. To this end, we propose the application of a fully data-driven approach to characterize nerve fiber architecture in 3D-PLI images using self-supervised representation learning. We introduce a 3D-Context Contrastive Learning (CL-3D) objective that utilizes the spatial neighborhood of texture examples across histological brain sections of a 3D reconstructed volume to sample positive pairs for contrastive learning. We combine this sampling strategy with specifically designed image augmentations to gain robustness to typical variations in 3D-PLI parameter maps. The approach is demonstrated for the 3D reconstructed occipital lobe of a vervet monkey brain. We show that extracted features are highly sensitive to different configurations of nerve fibers, yet robust to variations between consecutive brain sections arising from histological processing. We demonstrate their practical applicability for retrieving clusters of homogeneous fiber architecture, performing classification with minimal annotations, and query-based retrieval of characteristic components of fiber architecture such as U-fibers.

Figures (14)

• Figure 1: Overview of the 3D-PLI data acquisition. (A) Blockface images are taken for every section before slicing the mounted tissue block, providing a distortion-free reference for 3D volume reconstruction. ARTag markers positioned on the cryotome in the background are used for precise image alignment. (B) 3D-PLI measurement setup for the polarizing microscope (LMP-1) consisting of a coherent green light source, a rotating linear polarizer, a specimen stage, a stationary circular analyzer (quarter-wave retarder, linear polarizer) and a CCD camera to capture transmitted light intensities. (C) Example intensity profile recorded by a single pixel of the CCD camera at 9 polarizer rotation angles $\rho$. The profile can be described by a sinusodial curve, parameterized by three modalities: transmittance $I_T$, direction $\varphi$, and retardation $|\sin \delta| = \Delta I / I_T$.
• Figure 2: 3D reconstructed occipital lobe of the right hemisphere of a vervet monkey brain measured with 3D-PLI. (A) Localization of the occipital lobe on the surface of the 3D blockface reconstruction. Sections used for training (yellow), validation (red), and testing (blue) are color-coded. Numbers indicate section numbers. (B) 3D volume rendering of segmented cortical gray matter and white matter of the lobe. (C) 3D volume rendering for transmittance maps $I_T$ and (D) fiber orientation in HSV color space (hue: direction $\varphi$; saturation, brightness: inclination $\alpha$). Zoom-ins highlight the fiber architecture at the border between primary visual cortex (V1) and secondary visual cortex (V2). All volumes are masked at the pial boundary shown as a gray surface.
• Figure 3: Illustration of implemented 3D-PLI data augmentations for an example patch from the calcarine sulcus. Images show transmittance $I_T$, retardation $|\sin \delta|$ and fiber orientation in HSV color space (hue: direction $\varphi$; saturation, brightness: inclination $\alpha$). The colormap for retardation is scaled with a gamma correction for visibility. Parameters for the transmittance weighted model to compute fiber orientations are kept constant for all augmentations.
• Figure 4: Illustration of the proposed 3D context contrastive learning scheme. (A) Context sampling performed to obtain correlated views of similar nerve fiber architecture $(x_i, x_j)$ as (i) identical patches (Same), (ii) in-plane shifted patches on a circle with radius $r$ (CL-2D), or (iii) patches on a sphere with radius $r$ across sections (CL-3D). (B) Data augmentations $T$ for 3D-PLI are randomly applied to sampled patches to promote learning representations that are robust to typical variations in 3D-PLI measurements. Patches are visualized as transmittance $I_T$, retardation $|\sin \delta|$ and fiber orientation map (FOM) in HSV color space (hue: direction $\varphi$; saturation, brightness: inclination $\alpha$). (C) SimCLR contrastive learning framework chen2020a consisting of a ResNet encoder, hidden features $h_i$ and $h_j$, a fully connected MLP projection head, projections $z_i$ and $z_j$ and InfoNCE loss. (D) For inference, the trained encoder is applied on un-augmented patches $x$ to extract 3D-PLI texture features $h$. Whole sections are converted to feature maps using a sliding window approach. Two example feature maps are shown on top of transmittance maps for reference highlighting (i) U-fibers and (ii) primary visual cortex (V1).
• Figure 5: Comparison of different feature extraction methods under the linear evaluation protocol. A simple linear classifier is fitted on extracted features with an increasing number of labeled samples per class to classify texture patches as gray matter, white matter or background. With minimal samples per class provided, CL-3D and CL-2D features perform best, demonstrating highest robustness across the brain. Using the full number of samples per class, CL-3D, CL-2D and a combination of classical texture features (Combined) all match the performance of a ResNet-50 model trained specifically on this task, indicating a high-quality feature space of these methods. Shaded areas mark standard error over 50 independent fits of the classifier on randomly selected samples.