Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Autobiasing Event Cameras

Mehdi Sefidgar Dilmaghani, Waseem Shariff, Cian Ryan, Joseph Lemley, Peter Corcoran

TL;DR

An autonomous method to address challenges arising from severe lighting conditions in machine vision applications that use event cameras, using the neuromorphic YOLO-based face tracking module of a driver monitoring system as the event-based application to study.

Abstract

This paper presents an autonomous method to address challenges arising from severe lighting conditions in machine vision applications that use event cameras. To manage these conditions, the research explores the built in potential of these cameras to adjust pixel functionality, named bias settings. As cars are driven at various times and locations, shifts in lighting conditions are unavoidable. Consequently, this paper utilizes the neuromorphic YOLO-based face tracking module of a driver monitoring system as the event-based application to study. The proposed method uses numerical metrics to continuously monitor the performance of the event-based application in real-time. When the application malfunctions, the system detects this through a drop in the metrics and automatically adjusts the event cameras bias values. The Nelder-Mead simplex algorithm is employed to optimize this adjustment, with finetuning continuing until performance returns to a satisfactory level. The advantage of bias optimization lies in its ability to handle conditions such as flickering or darkness without requiring additional hardware or software. To demonstrate the capabilities of the proposed system, it was tested under conditions where detecting human faces with default bias values was impossible. These severe conditions were simulated using dim ambient light and various flickering frequencies. Following the automatic and dynamic process of bias modification, the metrics for face detection significantly improved under all conditions. Autobiasing resulted in an increase in the YOLO confidence indicators by more than 33 percent for object detection and 37 percent for face detection highlighting the effectiveness of the proposed method.

Autobiasing Event Cameras

TL;DR

An autonomous method to address challenges arising from severe lighting conditions in machine vision applications that use event cameras, using the neuromorphic YOLO-based face tracking module of a driver monitoring system as the event-based application to study.

Abstract

This paper presents an autonomous method to address challenges arising from severe lighting conditions in machine vision applications that use event cameras. To manage these conditions, the research explores the built in potential of these cameras to adjust pixel functionality, named bias settings. As cars are driven at various times and locations, shifts in lighting conditions are unavoidable. Consequently, this paper utilizes the neuromorphic YOLO-based face tracking module of a driver monitoring system as the event-based application to study. The proposed method uses numerical metrics to continuously monitor the performance of the event-based application in real-time. When the application malfunctions, the system detects this through a drop in the metrics and automatically adjusts the event cameras bias values. The Nelder-Mead simplex algorithm is employed to optimize this adjustment, with finetuning continuing until performance returns to a satisfactory level. The advantage of bias optimization lies in its ability to handle conditions such as flickering or darkness without requiring additional hardware or software. To demonstrate the capabilities of the proposed system, it was tested under conditions where detecting human faces with default bias values was impossible. These severe conditions were simulated using dim ambient light and various flickering frequencies. Following the automatic and dynamic process of bias modification, the metrics for face detection significantly improved under all conditions. Autobiasing resulted in an increase in the YOLO confidence indicators by more than 33 percent for object detection and 37 percent for face detection highlighting the effectiveness of the proposed method.

Paper Structure

This paper contains 16 sections, 2 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Event Camera Hardware and Biases
  3. Literature Review
  4. Conventional Camera Auto-exposure
  5. Event Camera Bias Setting
  6. Methodology
  7. Autobiasing Pipeline
  8. Building Blocks
  9. Event Camera
  10. Events Processor
  11. Face Detector
  12. Efficacy Metric Calculator
  13. Bias Controller
  14. Results and Discussion
  15. Limitations
  16. ...and 1 more sections

Figures (5)

  • Figure 1: The schematic of each pixel in event cameras w19.
  • Figure 2: The block diagram of the proposed autobiasing system.
  • Figure 3: Accumulation of event stream data in 2D frame matrices over time.m9
  • Figure 4: Autobiasing system flowchart; 'Bias Controller' enclosed within the dashed box.
  • Figure 5: Impact of autobiasing on face tracking metrics under 200 lux and flickering frequencies from 100 Hz to 500 Hz (Rows: flickering frequencies, Columns: face tracking metric, YOLO confidences on both object and face detection.)