Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Retina-Inspired Object Motion Segmentation for Event-Cameras

Victoria Clerico, Shay Snyder, Arya Lohia, Md Abdullah-Al Kaiser, Gregory Schwartz, Akhilesh Jaiswal, Maryam Parsa

TL;DR

The paper tackles object motion segmentation under ego-motion in event-camera data by introducing a retina-inspired, bio-plausible Object Motion Sensitivity (OMS) algorithm. Implemented as a lightweight, non-learning computation using center-surround Gaussian kernels and thresholding, OMS yields pixel-wise motion segmentation with only $80$ non-learnable parameters, enabling potential in-sensor processing. Evaluations on real and synthetic datasets demonstrate performance competitive with or superior to several state-of-the-art deep learning methods while dramatically reducing model complexity by $10^3$ to $10^6$×. This work highlights the practicality of neuroscience-inspired, domain-agnostic approaches for high-speed, low-bandwidth vision tasks and motivates integrating additional retinal functionalities for robust, in-sensor visual computation.

Abstract

Event-cameras have emerged as a revolutionary technology with a high temporal resolution that far surpasses standard active pixel cameras. This technology draws biological inspiration from photoreceptors and the initial retinal synapse. This research showcases the potential of additional retinal functionalities to extract visual features. We provide a domain-agnostic and efficient algorithm for ego-motion compensation based on Object Motion Sensitivity (OMS), one of the multiple features computed within the mammalian retina. We develop a method based on experimental neuroscience that translates OMS' biological circuitry to a low-overhead algorithm to suppress camera motion bypassing the need for deep networks and learning. Our system processes event data from dynamic scenes to perform pixel-wise object motion segmentation using a real and synthetic dataset. This paper introduces a bio-inspired computer vision method that dramatically reduces the number of parameters by $\text{10}^\text{3}$ to $\text{10}^\text{6}$ orders of magnitude compared to previous approaches. Our work paves the way for robust, high-speed, and low-bandwidth decision-making for in-sensor computations.

Retina-Inspired Object Motion Segmentation for Event-Cameras

TL;DR

The paper tackles object motion segmentation under ego-motion in event-camera data by introducing a retina-inspired, bio-plausible Object Motion Sensitivity (OMS) algorithm. Implemented as a lightweight, non-learning computation using center-surround Gaussian kernels and thresholding, OMS yields pixel-wise motion segmentation with only non-learnable parameters, enabling potential in-sensor processing. Evaluations on real and synthetic datasets demonstrate performance competitive with or superior to several state-of-the-art deep learning methods while dramatically reducing model complexity by to ×. This work highlights the practicality of neuroscience-inspired, domain-agnostic approaches for high-speed, low-bandwidth vision tasks and motivates integrating additional retinal functionalities for robust, in-sensor visual computation.

Abstract

Event-cameras have emerged as a revolutionary technology with a high temporal resolution that far surpasses standard active pixel cameras. This technology draws biological inspiration from photoreceptors and the initial retinal synapse. This research showcases the potential of additional retinal functionalities to extract visual features. We provide a domain-agnostic and efficient algorithm for ego-motion compensation based on Object Motion Sensitivity (OMS), one of the multiple features computed within the mammalian retina. We develop a method based on experimental neuroscience that translates OMS' biological circuitry to a low-overhead algorithm to suppress camera motion bypassing the need for deep networks and learning. Our system processes event data from dynamic scenes to perform pixel-wise object motion segmentation using a real and synthetic dataset. This paper introduces a bio-inspired computer vision method that dramatically reduces the number of parameters by to orders of magnitude compared to previous approaches. Our work paves the way for robust, high-speed, and low-bandwidth decision-making for in-sensor computations.
Paper Structure (8 sections, 1 figure, 4 tables, 1 algorithm)

This paper contains 8 sections, 1 figure, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Evaluation
  5. Datasets and Metrics
  6. Comparisons with state-of-the-art Methods
  7. Qualitative assessment
  8. Future Works and Discussion

Figures (1)

  • Figure 1: Comparative analysis between DVS event frames, ground truth mask and OMS for table, wall, floor, box, and fast sequences from EV-IMO mitrokhin2020evimo.