Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hardware-Algorithm Re-engineering of Retinal Circuit for Intelligent Object Motion Segmentation

Jason Sinaga, Victoria Clerico, Md Abdullah-Al Kaiser, Shay Snyder, Arya Lohia, Gregory Schwartz, Maryam Parsa, Akhilesh Jaiswal

TL;DR

The paper tackles object motion segmentation in ego-motion scenes using event-based DVS data and a biologically inspired Object Motion Sensitivity (OMS) circuit. It proposes hardware-algorithm co-design by re-engineering OMS through a software-validated OMS_CHANGE algorithm mapped to reconfigurable CMOS OMS circuits inside image sensors with runtime parameter reconfiguration and 3D integration considerations. Contributions include a software OMS implementation, a reconfigurable CMOS OMS circuit, a reconfigurable threshold circuit, and a reconfigurable OMS compute array, with Cadence 180 nm verification and 3D integration considerations. Results demonstrate that hardware-friendly kernel types and dynamic parameter tuning preserve segmentation performance while reducing circuit complexity, establishing a foundation for retinal circuits tailored to application-specific needs.

Abstract

Recent advances in retinal neuroscience have fueled various hardware and algorithmic efforts to develop retina-inspired solutions for computer vision tasks. In this work, we focus on a fundamental visual feature within the mammalian retina, Object Motion Sensitivity (OMS). Using DVS data from EV-IMO dataset, we analyze the performance of an algorithmic implementation of OMS circuitry for motion segmentation in presence of ego-motion. This holistic analysis considers the underlying constraints arising from the hardware circuit implementation. We present novel CMOS circuits that implement OMS functionality inside image sensors, while providing run-time re-configurability for key algorithmic parameters. In-sensor technologies for dynamical environment adaptation are crucial for ensuring high system performance. Finally, we verify the functionality and re-configurability of the proposed CMOS circuit designs through Cadence simulations in 180nm technology. In summary, the presented work lays foundation for hardware-algorithm re-engineering of known biological circuits to suit application needs.

Hardware-Algorithm Re-engineering of Retinal Circuit for Intelligent Object Motion Segmentation

TL;DR

The paper tackles object motion segmentation in ego-motion scenes using event-based DVS data and a biologically inspired Object Motion Sensitivity (OMS) circuit. It proposes hardware-algorithm co-design by re-engineering OMS through a software-validated OMS_CHANGE algorithm mapped to reconfigurable CMOS OMS circuits inside image sensors with runtime parameter reconfiguration and 3D integration considerations. Contributions include a software OMS implementation, a reconfigurable CMOS OMS circuit, a reconfigurable threshold circuit, and a reconfigurable OMS compute array, with Cadence 180 nm verification and 3D integration considerations. Results demonstrate that hardware-friendly kernel types and dynamic parameter tuning preserve segmentation performance while reducing circuit complexity, establishing a foundation for retinal circuits tailored to application-specific needs.

Abstract

Recent advances in retinal neuroscience have fueled various hardware and algorithmic efforts to develop retina-inspired solutions for computer vision tasks. In this work, we focus on a fundamental visual feature within the mammalian retina, Object Motion Sensitivity (OMS). Using DVS data from EV-IMO dataset, we analyze the performance of an algorithmic implementation of OMS circuitry for motion segmentation in presence of ego-motion. This holistic analysis considers the underlying constraints arising from the hardware circuit implementation. We present novel CMOS circuits that implement OMS functionality inside image sensors, while providing run-time re-configurability for key algorithmic parameters. In-sensor technologies for dynamical environment adaptation are crucial for ensuring high system performance. Finally, we verify the functionality and re-configurability of the proposed CMOS circuit designs through Cadence simulations in 180nm technology. In summary, the presented work lays foundation for hardware-algorithm re-engineering of known biological circuits to suit application needs.
Paper Structure (17 sections, 11 figures, 3 tables)

This paper contains 17 sections, 11 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Software implementation
  5. Reconfigurable CMOS OMS Circuit
  6. Reconfigurable Threshold Circuit
  7. Reconfigurable OMS Compute Array
  8. Results
  9. Software Simulations
  10. FILTER TYPE
  11. STRIDE
  12. TIME INTERVAL
  13. Hardware Simulations
  14. RECONFIGURABLE OMS COMPUTATION CIRCUIT
  15. RECONFIGURABLE THRESHOLD CIRCUIT
  16. ...and 2 more sections

Figures (11)

  • Figure 1: Original IRIS circuit with set center and surround regions from yin2022iris.
  • Figure 2: Schematic of the proposed OMS compute circuit where each pixel can be configured to be a part of the 'center' or 'surround' region.
  • Figure 3: Schematic of the reconfigurable threshold circuit based on programmable inverter's trip point.
  • Figure 4: Reconfigurable OMS Compute Array. (a) OMS compute circuit per pixel, (b) unit OMS compute cell used as the macro block in the 2D array, (c) reconfigurability controller circuit including wordline and bitlines, and (d) reconfigurable OMS compute 2D array.
  • Figure 5: A sample frame and label from EV-IMO mitrokhin2020evimo showing a DVS frame and the corresponding ground truth mask.
  • ...and 6 more figures