A Real-Time Event-Based Normal Flow Estimator

TL;DR

The paper tackles real-time estimation of generalized normal flow from asynchronous event streams. It introduces a real-time local event encoder and a pooling-based reformulation of Random Fourier Features encoding to replace quadratic-time adjacency computations, enabling high-throughput inference on GPUs. The approach preserves asynchronous predictions for each event, achieves substantial speedups (millions of flows per second on modern GPUs), and maintains competitive accuracy with a reduced feature dimension ($D=64$). This yields a practical, deployable solution for robotics and downstream event-based vision tasks, with an accompanying CUDA/Python release for broad usability.

Abstract

This paper presents a real-time, asynchronous, event-based normal flow estimator. It follows the same algorithm as Learning Normal Flow Directly From Event Neighborhoods, but with a more optimized implementation. The original method treats event slices as 3D point clouds, encodes each event's local geometry into a fixed-length vector, and uses a multi-layer perceptron to predict normal flow. It constructs representations by multiplying an adjacency matrix with a feature matrix, resulting in quadratic time complexity with respect to the number of events. In contrast, we leverage the fact that event coordinates are integers and reformulate the representation step as a pooling operation. This achieves the same effect as the adjacency matrix but with much lower computational cost. As a result, our method supports real-time normal flow prediction on event cameras. Our estimator uses 1 GB of CUDA memory and runs at 4 million normal flows per second on an RTX 3070, or 6 million per second on an RTX A5000. We release the CUDA implementation along with a Python interface at https://github.com/dhyuan99/VecKM_flow_cpp.