Dynamic Event-based Optical Identification and Communication

TL;DR

This work addresses the limitations of frame-based optical camera communication for dynamic scenarios by leveraging fast event-based cameras and a neuromorphic spiking neural network to perform sparse optical flow. The authors integrate DBSCAN clustering, a ΔSNU-based optical flow module, Kalman-filter tracking, and an 11-bit beacon encoding (start code $S_c$, payload $S_p$, parity $f(S_p)$) to enable simultaneous identification and tracking of moving beacons at $kHz$ rates. In simulation and on a hardware prototype, the approach achieves high message accuracy and bit accuracy for moving beacons, with demonstrated ranges and speeds surpassing prior OCC methods. The work thus advances dynamic optical identification by combining neuromorphic processing with event-based sensing to enable robust, real-time tracking and high-rate communication for asset monitoring and related applications.

Abstract

Optical identification is often done with spatial or temporal visual pattern recognition and localization. Temporal pattern recognition, depending on the technology, involves a trade-off between communication frequency, range and accurate tracking. We propose a solution with light-emitting beacons that improves this trade-off by exploiting fast event-based cameras and, for tracking, sparse neuromorphic optical flow computed with spiking neurons. The system is embedded in a simulated drone and evaluated in an asset monitoring use case. It is robust to relative movements and enables simultaneous communication with, and tracking of, multiple moving beacons. Finally, in a hardware lab prototype, we demonstrate for the first time beacon tracking performed simultaneously with state-of-the-art frequency communication in the kHz range.

Figures (9)

• Figure 1: Architectural diagram of our system. The beacon's light is detected by the sensor as events. Events are processed to track the beacons and further decode the transmitted messages. The event array block shows a snapshot of recorded events.
• Figure 2: A valid sequence, decoded from blinking transitions.
• Figure 3: SNN for sparse optical flow.a. Events from camera at each input location are processed by 32 $\Delta$SNU units, each with specific synaptic delays. b. The magnitudes of synaptic delays are attuned to 8 different movement angles (spatial gradient of delays) and 4 different speeds (different magnitudes of delays), schematically indicated by red arrows.
• Figure 4: Tracking steps.a. Reading optical flow (red arrow) at the track's location. b. Prediction of the Kalman state via the track's location and the optical flow value. c. Tracks are assigned to a target in its oriented neighborhood, based upon the track's motion. d. The track's state, its size and its polarity are updated with the paired target's properties.
• Figure 5: Decoupled loops: The tracking loop has a much lower and fixed frequency to maintain efficiency, while the decoding loop has the same frequency as the emitter to be able to decode the received signal.