Enhancing Visual Place Recognition via Fast and Slow Adaptive Biasing in Event Cameras

TL;DR

This work addresses the sensitivity of event-camera VPR performance to bias parameters by introducing a novel fast-and-slow closed-loop controller that first rapidly stabilizes the event rate via refractory-period adjustments and then gradually tunes pixel bandwidth and event thresholds when needed. The approach is validated on the QCR-Fast-and-Slow dataset, showing superior or on-par performance with state-of-the-art baselines, particularly under challenging lighting changes. Key contributions include the two-tier adaptive biasing mechanism, a publicly available indoor dataset, and ablation studies that demonstrate the benefit of combining fast and slow adaptations for robust VPR. The findings underscore the practical impact of task-aligned bias control for event cameras and point to future work integrating with SNNs and multimodal sensors for even more resilient perception in robotics.

Abstract

Event cameras are increasingly popular in robotics due to beneficial features such as low latency, energy efficiency, and high dynamic range. Nevertheless, their downstream task performance is greatly influenced by the optimization of bias parameters. These parameters, for instance, regulate the necessary change in light intensity to trigger an event, which in turn depends on factors such as the environment lighting and camera motion. This paper introduces feedback control algorithms that automatically tune the bias parameters through two interacting methods: 1) An immediate, on-the-fly \textit{fast} adaptation of the refractory period, which sets the minimum interval between consecutive events, and 2) if the event rate exceeds the specified bounds even after changing the refractory period repeatedly, the controller adapts the pixel bandwidth and event thresholds, which stabilizes after a short period of noise events across all pixels (\textit{slow} adaptation). Our evaluation focuses on the visual place recognition task, where incoming query images are compared to a given reference database. We conducted comprehensive evaluations of our algorithms' adaptive feedback control in real-time. To do so, we collected the QCR-Fast-and-Slow dataset that contains DAVIS346 event camera streams from 366 repeated traversals of a Scout Mini robot navigating through a 100 meter long indoor lab setting (totaling over 35km distance traveled) in varying brightness conditions with ground truth location information. Our proposed feedback controllers result in superior performance when compared to the standard bias settings and prior feedback control methods. Our findings also detail the impact of bias adjustments on task performance and feature ablation studies on the fast and slow adaptation mechanisms.

Figures (9)

• Figure 1: Closed-loop biasing of event cameras. We perform feedback control on event camera biases to obtain improved performance for the visual place recognition downstream task. We achieve this by varying refractory period, pixel bandwidth, and event threshold parameters in real-time. As illustrated, we modulate these device parameters to monitor and maintain event-rate from an onboard event-camera mounted on a mobile robot.
• Figure 2: Dynamic bias adjustments for differing brightness conditions. The top two graphs display the fine-tuning of bias parameters over time, while the bottom graphs show the corresponding event rates. The refractory period (blue) is constantly adjusted, while the other bias parameters are only adjusted if the refractory period changes alone do not suffice. On the left, under high brightness conditions, the biases are adjusted less frequently, resulting in a relatively stable event rate. Conversely, on the right, for low brightness conditions, there is more frequent fine-tuning of biases, which correlates with the more variable event rate observed. This illustrates how bias parameter management is critical for adapting to different luminosity levels and maintaining consistent event detection rates.
• Figure 3: Refractory period. Left: A short refractory period (black lines indicate the refractory period) allows more frequent firing of events (as indicated by the red and blue lines in the bottom plots), resulting in a signal that closely follows the input stimulus. Right: An extended refractory period limits the firing rate, leading to fewer events and a signal that may omit rapid fluctuations of the input stimulus.
• Figure 4: Pixel bandwidth. Left: A high bandwidth allows high frequency signal to pass through, which leads to higher event rates. Right: A low bandwidth filters high frequency components of the signal, leading to a low frequency signal and consequently lower event rate.
• Figure 5: Event threshold. Left: A low event threshold setting (indicated by dashed horizontal lines) results in a high event rate (frequent events in the bottom plot), capturing most changes in the input signal. Right: A high threshold setting results in a lower event rate (much fewer events, as indicated by the sparse red and blue lines), only responding to larger variations in the input signal.