E-VLC: A Real-World Dataset for Event-based Visible Light Communication And Localization

TL;DR

The paper introduces E-VLC, the first public dataset for event-camera visible-light communication and localization, featuring synchronized event and frame data, precise ground-truth poses via hardware triggers, and LED bounding-box annotations across varied indoor/outdoor scenarios. It also proposes a Contrast Maximization-based motion-compensation pipeline to improve LED detection and camera localization when LEDs flicker and scene motion violate brightness constancy. Key contributions include the dataset design (hardware, scenarios, annotations) and a motion-aware decoding method that demonstrates advantages of LED-based localization over frame-based AR markers under challenging lighting and motion conditions. The work provides a practical benchmark for joint motion-related vision tasks and LED signal decoding on mobile/edge devices, enabling broader adoption of event cameras in VLC and localization tasks.

Abstract

Optical communication using modulated LEDs (e.g., visible light communication) is an emerging application for event cameras, thanks to their high spatio-temporal resolutions. Event cameras can be used simply to decode the LED signals and also to localize the camera relative to the LED marker positions. However, there is no public dataset to benchmark the decoding and localization in various real-world settings. We present, to the best of our knowledge, the first public dataset that consists of an event camera, a frame camera, and ground-truth poses that are precisely synchronized with hardware triggers. It provides various camera motions with various sensitivities in different scene brightness settings, both indoor and outdoor. Furthermore, we propose a novel method of localization that leverages the Contrast Maximization framework for motion estimation and compensation. The detailed analysis and experimental results demonstrate the advantages of LED-based localization with events over the conventional AR-marker--based one with frames, as well as the efficacy of the proposed method in localization. We hope that the proposed dataset serves as a future benchmark for both motion-related classical computer vision tasks and LED marker decoding tasks simultaneously, paving the way to broadening applications of event cameras on mobile devices. https://woven-visionai.github.io/evlc-dataset

Figures (9)

• Figure 1: We propose the E-VLC dataset and a novel method that compensates motion to improve the localization accuracy. The dataset consists of synchronized events, frames, and GT poses as well as bounding boxes, in various recording settings.
• Figure 2: Data recording setup. The mobile recorder consists of two cameras ($1280\times1024$-pix frame and $1280\times720$-pix event) that are synchronized via a trigger box and a tablet. The trigger sends the signal to the external motion capture. We record the data both in outdoor and indoor environments for different scene luminance. Best viewed online.
• Figure 3: LED markers on different objects. Orange circles indicate the LEDs, each of which transmits different ID signals. Yellow circles are markers for the motion capture system.
• Figure 4: Examples of the dataset. Each row describes examples of different motion patterns, scene luminances, and camera sensitivities. The "Low" sensitivity sequences (first and last rows) only record markers, while others record markers and edges. Best viewed online (enlarged).
• Figure 5: Proposed motion compensation pipeline. With the low camera sensitivity (the top diagram), LED information is decoded directly from events. With the middle or high sensitivities (bottom diagram), we leverage the motion estimation and compensation algorithms to improve the localization accuracy.