NeuroFly: A framework for whole-brain single neuron reconstruction

TL;DR

NeuroFly is introduced, a validated framework for large-scale automatic single neuron reconstruction and its efficiency is validated using in-house datasets that include a variety of challenging scenarios, such as dense arborizations, weak axons, images with contamination.

Abstract

Neurons, with their elongated, tree-like dendritic and axonal structures, enable efficient signal integration and long-range communication across brain regions. By reconstructing individual neurons' morphology, we can gain valuable insights into brain connectivity, revealing the structure basis of cognition, movement, and perception. Despite the accumulation of extensive 3D microscopic imaging data, progress has been considerably hindered by the absence of automated tools to streamline this process. Here we introduce NeuroFly, a validated framework for large-scale automatic single neuron reconstruction. This framework breaks down the process into three distinct stages: segmentation, connection, and proofreading. In the segmentation stage, we perform automatic segmentation followed by skeletonization to generate over-segmented neuronal fragments without branches. During the connection stage, we use a 3D image-based path following approach to extend each fragment and connect it with other fragments of the same neuron. Finally, human annotators are required only to proofread the few unresolved positions. The first two stages of our process are clearly defined computer vision problems, and we have trained robust baseline models to solve them. We validated NeuroFly's efficiency using in-house datasets that include a variety of challenging scenarios, such as dense arborizations, weak axons, images with contamination. We will release the datasets along with a suite of visualization and annotation tools for better reproducibility. Our goal is to foster collaboration among researchers to address the neuron reconstruction challenge, ultimately accelerating advancements in neuroscience research. The dataset and code are available at https://github.com/beanli161514/neurofly

Figures (8)

• Figure 1: A mouse neuron reconstructed from fluorescence microscope images at a resolution of $1~\mu m/\text{voxel}$. (a) Image of a whole mouse brain and a reconstructed neuron. The color of the neuron segments indicates the structural distance, defined as the number of branch points encountered along the shortest path to the soma. The image size is $14000 \times 10000 \times 14000~\mu m^3$. (b) Arborization of the axon at the surface of the brain cortex, with an image size of $512 \times 512 \times 512~\mu m^3$. (c) Soma and dendrites of the neuron, with an image size of $384 \times 384 \times 384~\mu m^3$.
• Figure 2: Illustration of NeuroFly's 3 stages. (a) Image of axon arborization used as example, with a size of $800 \times 600 \times 300 ~ \mu m^3$. (b) Result of the segmentation stage. Red lines represent neurites' centerlines extracted by the foreground segmentation followed by skeletonization. (c) Result of the connection stage. Orange lines represent the trajectories of agents starting off at the endpoints of each incomplete segments. (d) Kinetics and trajectory of the agent. $\vec{n}_1, \vec{n}_2, ~ \text{and} ~ \vec{t}~$ composes an rotation-minimizing frame, where $\vec{t}~$ denotes the tangent direction of the agent's movement. (e) Final reconstruction of the axon arborization. Neuron segments are shifted 10 voxels from the image and colored according to their connectivity for better visualization.
• Figure 3: Examples of contaminated images, with contamination indicated by red arrows. The middle image shows a sheet-like matter adhering to the brain's surface that might be a piece of meninges. The other two images contain blood vessels.
• Figure 4: illustration of 3D image-based path following. The agent is denoted by red dot centering at its local image volume. Specifically, the directions of the x, y, and z axes of the local image align respectively with $\mathbf{n}_1$, $\mathbf{n}_2$, and $\mathbf{t}$.
• Figure 5: Illustration of data augmentation based on image fusion and histogram matching.