Joint Learning Neuronal Skeleton and Brain Circuit Topology with Permutation Invariant Encoders for Neuron Classification

TL;DR

This paper reprocess and release two new datasets for neuron classification task from volume electron microscopy images of human brain cortex and Drosophila brain, and proposes NeuNet, a framework that combines morphological information of neurons obtained from skeleton and topological information between neurons obtained from neural circuit.

Abstract

Determining the types of neurons within a nervous system plays a significant role in the analysis of brain connectomics and the investigation of neurological diseases. However, the efficiency of utilizing anatomical, physiological, or molecular characteristics of neurons is relatively low and costly. With the advancements in electron microscopy imaging and analysis techniques for brain tissue, we are able to obtain whole-brain connectome consisting neuronal high-resolution morphology and connectivity information. However, few models are built based on such data for automated neuron classification. In this paper, we propose NeuNet, a framework that combines morphological information of neurons obtained from skeleton and topological information between neurons obtained from neural circuit. Specifically, NeuNet consists of three components, namely Skeleton Encoder, Connectome Encoder, and Readout Layer. Skeleton Encoder integrates the local information of neurons in a bottom-up manner, with a one-dimensional convolution in neural skeleton's point data; Connectome Encoder uses a graph neural network to capture the topological information of neural circuit; finally, Readout Layer fuses the above two information and outputs classification results. We reprocess and release two new datasets for neuron classification task from volume electron microscopy(VEM) images of human brain cortex and Drosophila brain. Experiments on these two datasets demonstrated the effectiveness of our model with accuracy of 0.9169 and 0.9363, respectively. Code and data are available at: https://github.com/WHUminghui/NeuNet.

Figures (8)

• Figure 1: Neuron connectivity (orange) and morphology (blue) in a Drosophila (fruit fly) brain. For each neuron, its connectivity can be represented by a graph. Its morphology can be described by a skeleton that is a set of 3D points. Both of them can reveal neuron's functions and roles.
• Figure 2: Architecture of proposed NeuNet. On the one hand, the skeleton data of neurons are input into Skeleton Encoder, and Sampling operation samples the points in the skeleton's point set using FPS. Grouping operation divides the attached points into groups with the sampled points as the center, and ConvBlock extracts the features on these group's points and fuses them using max pooling to obtain the group's global features. The above process is repeated until the skeleton's global features are obtained. On the other hand, GNN-based Connectome Encoder mines the topological features of the neural circuit through the information interaction between nodes and neighboring nodes. Finally, Readout Layer fuses the above two features and outputs the classification results with an MLP.
• Figure 3: HemiBrain neuron skeletons of KC (Kenyon Cells), LHAV(Lateral Horn Anterior Ventral cell cluster), MC(Medulla Columnar), SLP(Superior lateral protocerebrum), and WED(Wedge). The first column are neurons (blue) from the training set, the rest of the columns are NeuNet's prediction cases on test, and the gray background is Drosophila brain. NeuNet learned the morphology of the neurons, including the "L" shaped structure of the KC class.
• Figure 4: The visualizations using t-SNE tSNE of neuron representation learned by various methods. Where each of color represents a specific class. Since there are up to 191 class of HemiBrain, different classes of point may be given the same color.
• Figure 5: Query neurons(red) and retrieved neurons(green) using NeuNet. The gray-shaded background represents the Drosophila brain, and the red shaded background indicates the region with the highest synaptic connectivity for the query neuron. Query neuron IDs and finer-grained categories are provided by hemiBrain. The subcategories of the top 10 retrieved neurons remain consistent with the query neurons(not shown in this figure), without the utilization of finer-grained subcategory information in NeuNet.

Theorems & Definitions (4)

• Definition 1: Permutation action $\pi$
• Definition 2: Permutation invariant function
• Theorem 1
• Theorem 2