Fast Event-based Optical Flow Estimation by Triplet Matching

TL;DR

This paper addresses the need for fast, lightweight optical-flow estimation on event cameras by introducing a triplet-matching algorithm inspired by neuroscience. It provides an incremental, event-by-event estimator and a batch extension, both based on correlating triplets of events within space-time neighborhoods to infer local motion; the flow for an event is the Gaussian-weighted average of triplet velocities, with indexing structures to enable efficient searches. Empirically, the method delivers comparable accuracy to batch-based approaches on standard benchmarks while achieving real-time CPU-only performance (>10 kHz) and demonstrating robustness to practical constraints. The work highlights a path toward real-time, incremental motion estimation on resource-constrained devices and offers insights into neuromorphic-inspired, logic-based processing of event data.

Abstract

Event cameras are novel bio-inspired sensors that offer advantages over traditional cameras (low latency, high dynamic range, low power, etc.). Optical flow estimation methods that work on packets of events trade off speed for accuracy, while event-by-event (incremental) methods have strong assumptions and have not been tested on common benchmarks that quantify progress in the field. Towards applications on resource-constrained devices, it is important to develop optical flow algorithms that are fast, light-weight and accurate. This work leverages insights from neuroscience, and proposes a novel optical flow estimation scheme based on triplet matching. The experiments on publicly available benchmarks demonstrate its capability to handle complex scenes with comparable results as prior packet-based algorithms. In addition, the proposed method achieves the fastest execution time (> 10 kHz) on standard CPUs as it requires only three events in estimation. We hope that our research opens the door to real-time, incremental motion estimation methods and applications in real-world scenarios.

Figures (4)

• Figure 1: Runtime vs. accuracy comparison for various event-based optical flow estimation methods. Results are on outdoor data of the MVSEC benchmark Zhu18rss (see also \ref{['tab:main_mvsec']}). Accuracy is measured based on Average Endpoint Error (AEE). Shiba Shiba22eccv(a) and Shiba Shiba22eccv(b) denote optimization-based and DNN-based results, respectively.
• Figure 2: Triplet matchig algorithm. The algorithm seeks spatially and temporally neighboring events in an event-by-event manner, and provides event-based flow $\mathbf{f}_k$. For ease of visualization we only show the search in $x$ and $t$, but it is actually carried out in $x,y$ and $t$. Note this is an example of batch estimation given the input events.
• Figure 3: Optical flow results on MVSEC data.
• Figure 4: Effect of pixel quantization on ECD data. In the top row the motion is dominantly horizontal, whereas in the bottom row it is vertical, as can be seen by the thickness of the edges (left) and the velocity distributions (right).