Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Real-time Neuron Segmentation for Voltage Imaging

Yosuke Bando, Ramdas Pillai, Atsushi Kajita, Farhan Abdul Hakeem, Yves Quemener, Hua-an Tseng, Kiryl D. Piatkevich, Changyang Linghu, Xue Han, Edward S. Boyden

TL;DR

A fast neuron segmentation method able to detect multiple, potentially overlapping, spiking neurons from noisy video frames is proposed and implemented, and a data processing pipeline incorporating the proposed segmentation method along with GPU-accelerated motion correction is implemented.

Abstract

In voltage imaging, where the membrane potentials of individual neurons are recorded at from hundreds to thousand frames per second using fluorescence microscopy, data processing presents a challenge. Even a fraction of a minute of recording with a limited image size yields gigabytes of video data consisting of tens of thousands of frames, which can be time-consuming to process. Moreover, millisecond-level short exposures lead to noisy video frames, obscuring neuron footprints especially in deep-brain samples where noisy signals are buried in background fluorescence. To address this challenge, we propose a fast neuron segmentation method able to detect multiple, potentially overlapping, spiking neurons from noisy video frames, and implement a data processing pipeline incorporating the proposed segmentation method along with GPU-accelerated motion correction. By testing on existing datasets as well as on new datasets we introduce, we show that our pipeline extracts neuron footprints that agree well with human annotation even from cluttered datasets, and demonstrate real-time processing of voltage imaging data on a single desktop computer for the first time.

Real-time Neuron Segmentation for Voltage Imaging

TL;DR

A fast neuron segmentation method able to detect multiple, potentially overlapping, spiking neurons from noisy video frames is proposed and implemented, and a data processing pipeline incorporating the proposed segmentation method along with GPU-accelerated motion correction is implemented.

Abstract

In voltage imaging, where the membrane potentials of individual neurons are recorded at from hundreds to thousand frames per second using fluorescence microscopy, data processing presents a challenge. Even a fraction of a minute of recording with a limited image size yields gigabytes of video data consisting of tens of thousands of frames, which can be time-consuming to process. Moreover, millisecond-level short exposures lead to noisy video frames, obscuring neuron footprints especially in deep-brain samples where noisy signals are buried in background fluorescence. To address this challenge, we propose a fast neuron segmentation method able to detect multiple, potentially overlapping, spiking neurons from noisy video frames, and implement a data processing pipeline incorporating the proposed segmentation method along with GPU-accelerated motion correction. By testing on existing datasets as well as on new datasets we introduce, we show that our pipeline extracts neuron footprints that agree well with human annotation even from cluttered datasets, and demonstrate real-time processing of voltage imaging data on a single desktop computer for the first time.
Paper Structure (11 sections, 3 equations, 9 figures, 3 tables)

This paper contains 11 sections, 3 equations, 9 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Pipeline Overview
  3. Neuron Segmentation
  4. Motivation and Design
  5. Time Segment Summarization
  6. Spiking Pixel Identification
  7. Neuron Footprint Reconstruction
  8. Evaluation
  9. Speed Evaluation
  10. Accuracy Evaluation
  11. Conclusion

Figures (9)

  • Figure 1: Voltage imaging data processing pipeline. Our pipeline processes an input video in real time, meaning that the processing time is shorter than the recording time. As an example, given a video recorded in 13.5 sec capturing 10,000 frames at 741 fps, the three stages of the pipeline spend 5.5 sec, 6.9 sec, and 0.1 sec, respectively, totaling 12.5 sec, which is shorter than the recording time of 13.5 sec.
  • Figure 2: Summary images used in VolPy VolPy for the input video in Figure \ref{['fig:pipeline']}.
  • Figure 3: Proposed segmentation subpipeline. The depth direction represents the time axis.
  • Figure 4: Max-median filter.
  • Figure 6: Our lightweight U-Net configuration with a small input size of 64x64. The diagram convention follows the original U-Net work UNet.
  • ...and 4 more figures