Lattice-allocated Real-time Line Segment Feature Detection and Tracking Using Only an Event-based Camera

TL;DR

This work tackles real-time line segment detection and tracking using only a high-rate event-based camera. It introduces SCARF, a velocity-invariant, lattice-allocated representation that buffers events in blocks, enabling a three-thread pipeline (SCARF, Detection, Tracking) to operate in parallel without frame data. Line segments are initialized inside blocks via a fitting score and then tracked by perturbing endpoints across blocks, achieving real-time performance (e.g., detection >200 Hz, tracking >400 Hz) on high-resolution data and outperforming state-of-the-art baselines in accuracy and robustness. The approach yields dense, longer-lived line segments suitable for real-world robotics tasks and SLAM, with open-source code and supplementary materials supporting replication and deployment.

Abstract

Line segment extraction is effective for capturing geometric features of human-made environments. Event-based cameras, which asynchronously respond to contrast changes along edges, enable efficient extraction by reducing redundant data. However, recent methods often rely on additional frame cameras or struggle with high event rates. This research addresses real-time line segment detection and tracking using only a modern, high-resolution (i.e., high event rate) event-based camera. Our lattice-allocated pipeline consists of (i) velocity-invariant event representation, (ii) line segment detection based on a fitting score, (iii) and line segment tracking by perturbating endpoints. Evaluation using ad-hoc recorded dataset and public datasets demonstrates real-time performance and higher accuracy compared to state-of-the-art event-only and event-frame hybrid baselines, enabling fully stand-alone event camera operation in real-world settings.

Figures (13)

• Figure 1: The proposed lattice-allocated real-time line segment detection and tracking. The (a) events are stored into lattice storage by (b) a velocity-invariant method SCARF. Line segments are (c) initialized within a block using only events from the active region of the lattice storage (Detection). Subsequently, their positions are updated both inside and outside the block using events from the active and inactive regions (Tracking).
• Figure 2: Parallelized pipeline overview of lattice-allocated line segment detection and tracking
• Figure 3: Overview of SCARF to store and visualize events with a lattice structure. (a) The lattice structure is defined with active and inactive regions. (b) Each block maintains a preallocated FIFO buffer for storing events. (c) For visualization, an image-like representation can be generated by assigning a pixel intensity $I$ to all "active" events within the buffer. (d) SCARF buffer update process. At $t_0$, the upper buffer is filled with inactive events. At $t_1$, it is "excited" by active events, while the bottom buffer is "inhibited". When the edge leaves the upper active region ($t_2$), the upper buffer becomes inhibited and the bottom buffer is excited. At $t_3$, the upper buffer is fully inactive, and inhibition begins in the bottom buffer.
• Figure 4: Definition of fitting score
• Figure 5: Perturbation of endpoints for tracking