Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Real-Time Optical Communication Using Event-Based Vision with Moving Transmitters

Harmeet Dhillon, Pranay Katyal, Brendan Long, Rohan Walia, Matthew Cleaveland, Kevin Leahy

Abstract

In multi-robot systems, traditional radio frequency (RF) communication struggles with contention and jamming. Optical communication offers a strong alternative. However, conventional frame-based cameras suffer from limited frame rates, motion blur, and reduced robustness under high dynamic range lighting. Event cameras support microsecond temporal resolution and high dynamic range, making them extremely sensitive to scene changes under fast relative motion with an optical transmitter. Leveraging these strengths, we develop a complete optical communication system capable of tracking moving transmitters and decoding messages in real time. Our system achieves over $95\%$ decoding accuracy for text transmission during motion by implementing a Geometry-Aware Unscented Kalman Filter (GA-UKF), achieving 7x faster processing speed compared to the previous state-of-the-art method, while maintaining equivalent tracking accuracy at transmitting frequencies $\geq$ 1 kHz.

Real-Time Optical Communication Using Event-Based Vision with Moving Transmitters

Abstract

In multi-robot systems, traditional radio frequency (RF) communication struggles with contention and jamming. Optical communication offers a strong alternative. However, conventional frame-based cameras suffer from limited frame rates, motion blur, and reduced robustness under high dynamic range lighting. Event cameras support microsecond temporal resolution and high dynamic range, making them extremely sensitive to scene changes under fast relative motion with an optical transmitter. Leveraging these strengths, we develop a complete optical communication system capable of tracking moving transmitters and decoding messages in real time. Our system achieves over decoding accuracy for text transmission during motion by implementing a Geometry-Aware Unscented Kalman Filter (GA-UKF), achieving 7x faster processing speed compared to the previous state-of-the-art method, while maintaining equivalent tracking accuracy at transmitting frequencies 1 kHz.
Paper Structure (22 sections, 15 equations, 7 figures, 1 table, 1 algorithm)

This paper contains 22 sections, 15 equations, 7 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. Transmitter tracking using blob model
  5. Geometry-Aware State Estimation
  6. Geometry-Aware Unscented Kalman Filter
  7. Geometry-aware state estimation
  8. Geometry of Ellipse Parameters
  9. Product Manifold Approximation
  10. State Space Formulation
  11. Axis Length Prediction
  12. GA-UKF Implementation
  13. Process Model for GA-UKF
  14. Temporally Weighted Measurements for GA-UKF
  15. Semi-Axis Length Update
  16. ...and 7 more sections

Figures (7)

  • Figure 1: Hardware experimental setup. Bottom: Prophesee EVK4 event camera capturing a strobing LED mounted on the drone. Inset: Close-up view of the drone with the LED.
  • Figure 2: Left: Estimated blob orientation using linear (Euclidean) mapping (red) versus an $\mathbb{S}^1$-wrapped update (blue), with the ground-truth orientation shown in green. Right: Illustration of the corresponding rotation trajectories. Dashed curves denote intermediate states and solid ellipses denote final estimates. The Euclidean update follows a longer rotation path, whereas the $\mathbb{S}^1$-wrapped update follows the shorter, geometrically consistent path. Dynamic visualization is presented in the supplementary video.
  • Figure 3: Left: GA-UKF tracks LED position and shape amid motion. The green ellipse represents all events in one batch update, the red circle represents true LED position, and the purple ellipse shows the estimated blob. Right: Continuous-time signal reconstruction using dual-threshold hysteresis, robustly recovering the binary sequence despite motion and noise.
  • Figure 4: Event pipeline: spatially filtered events are buffered, tracked with GA-UKF, and decoded via a shared buffer to reconstruct binary data in real time.
  • Figure 5: Processing time comparison between GA-UKF and EKF from async_blob_wang at varying signal frequencies.
  • ...and 2 more figures