MoCom: Motion-based Inter-MAV Visual Communication Using Event Vision and Spiking Neural Networks

TL;DR

MoCom tackles the challenge of reliable inter-MAV communication in cluttered and adversarial environments by signaling information through deliberate motion patterns captured by event cameras. It integrates event-frame based motion segmentation with a lightweight Spiking Neural Network (EventMAVNet) and an online Integrated Segmentation and Recognition (IMSR) decoding pipeline to reliably decode motion-based messages. The approach achieves high action recognition accuracy (up to 95+%) with ultra-low latency (≈1.26 ms per 10 samples) and demonstrates successful in-flight decoding and navigation in flight tests, highlighting energy efficiency and resilience to interference. This work offers a practical vision-based communication alternative for decentralized MAV swarms, enabling robust, covert coordination in GPS-denied or spectrum-constrained settings.

Abstract

Reliable communication in Micro Air Vehicle (MAV) swarms is challenging in environments, where conventional radio-based methods suffer from spectrum congestion, jamming, and high power consumption. Inspired by the waggle dance of honeybees, which efficiently communicate the location of food sources without sound or contact, we propose a novel visual communication framework for MAV swarms using motion-based signaling. In this framework, MAVs convey information, such as heading and distance, through deliberate flight patterns, which are passively captured by event cameras and interpreted using a predefined visual codebook of four motion primitives: vertical (up/down), horizontal (left/right), left-to-up-to-right, and left-to-down-to-right, representing control symbols (``start'', ``end'', ``1'', ``0''). To decode these signals, we design an event frame-based segmentation model and a lightweight Spiking Neural Network (SNN) for action recognition. An integrated decoding algorithm then combines segmentation and classification to robustly interpret MAV motion sequences. Experimental results validate the framework's effectiveness, which demonstrates accurate decoding and low power consumption, and highlights its potential as an energy-efficient alternative for MAV communication in constrained environments.

Figures (13)

• Figure 1: Overview of the proposed motion-based inter-MAV visual communication (MoCom) framework for MAV swarms. Traditional radio-based communication can be easily disrupted in constrained environments. Inspired by the waggle dance of honeybees, a motion-based signaling approach, where MAVs convey information through deliberate flight patterns. These motions are captured by event cameras. The event stream is then processed using a biologically inspired pipeline that segments the motion and classifies it using a spiking neural network. This paradigm offers a decentralized and interference-resilient alternative to conventional communication in MAV swarms.
• Figure 2: Several common visual communication codes are presented. Specifically, we define four types of communication codes (action : "start", action : "end", action : "1", action : "0"). A single event MAV motion is segmented into $F$-th event block based on an equal event count strategy, and the accumulated events are used to construct $F$ event frames. Red color represents positive events, green color represents negative events, and the yellow regions indicate the overlap between positive and negative events.
• Figure 3: MAV motion event stream. This diagram depicts the motion flow of a complete visual message, where $A_k$ represents predefined MAV motions, $S_e$ denotes the MAV static state (Empty signal), and $t_n$ indicates the termination timestamp of each MAV motion. From a time-domain perspective, an event signal accumulates progressively from left to right. The final "end" motion signifies the stop of the visual signal. Besides, the $x$-axis and $y$-axis represent the spatial domain of the event camera, while the time axis $t$ represents the time domain of the event camera.
• Figure 4: EventMAVNet: MAV action recognition model. This diagram illustrates the complete recognition process of the MAV event stream, implemented as a simple yet effective five-layer spiking neural network, encompassing four main components: event data pre-processing, feature extraction stage, feature encoding stage, and voting classification stage.
• Figure 5: MAV motion data collection system. The figure lists the necessary data collection sensors and key processing devices.