MAVR-Net: Robust Multi-View Learning for MAV Action Recognition with Cross-View Attention

TL;DR

MAVR-Net tackles robust MAV action recognition by fusing RGB appearance, optical-flow motion, and segmentation-based shape cues through a unified multi-view framework. It deploys a multi-view feature pyramid for cross-scale fusion and a Transformer-inspired cross-view attention module to model inter-modal dependencies, complemented by a multi-view alignment loss and attention regularization to stabilize training. Empirical results on a MAV action dataset show near-sota accuracy across short, medium, and long distances (up to 97.8% accuracy) and strong training stability, with ablations validating the contributions of MVFPM, CVAM, and L_align. This approach enhances robust perception and potential secure, decentralized MAV swarm coordination by leveraging complementary visual cues in challenging aerial environments.

Abstract

Recognizing the motion of Micro Aerial Vehicles (MAVs) is crucial for enabling cooperative perception and control in autonomous aerial swarms. Yet, vision-based recognition models relying only on RGB data often fail to capture the complex spatial temporal characteristics of MAV motion, which limits their ability to distinguish different actions. To overcome this problem, this paper presents MAVR-Net, a multi-view learning-based MAV action recognition framework. Unlike traditional single-view methods, the proposed approach combines three complementary types of data, including raw RGB frames, optical flow, and segmentation masks, to improve the robustness and accuracy of MAV motion recognition. Specifically, ResNet-based encoders are used to extract discriminative features from each view, and a multi-scale feature pyramid is adopted to preserve the spatiotemporal details of MAV motion patterns. To enhance the interaction between different views, a cross-view attention module is introduced to model the dependencies among various modalities and feature scales. In addition, a multi-view alignment loss is designed to ensure semantic consistency and strengthen cross-view feature representations. Experimental results on benchmark MAV action datasets show that our method clearly outperforms existing approaches, achieving 97.8\%, 96.5\%, and 92.8\% accuracy on the Short MAV, Medium MAV, and Long MAV datasets, respectively.

Figures (10)

• Figure 1: Overview of the proposed MAVR-Net architecture compared with traditional single-view models. (Left) Single-branch networks process only RGB frames, making them susceptible to noise and motion ambiguity. (Right) The proposed multi-view learning framework extracts complementary features from RGB, optical flow, and segmentation modalities using parallel branches, followed by feature fusion and cross-view attention to enhance motion discrimination and robustness across diverse flight conditions.
• Figure 2: Representative motion trajectories of the four MAV action classes: "vShape", "inv_vShape", "left_right", and "up_right". These motion types correspond to distinct flight trajectories recorded by the RGB camera and serve as the labeled action categories in the MAVR-Net dataset for training and evaluation.
• Figure 3: The whole pipeline of multi-view learning framework. From left to right, the raw MAV video action data is processed by multi-view extraction, multi-view feature pyramid process and etc. Besides, $Q$, $K$, and $V$ represent the Query, Key, and Value feature matrices used in the multi-head attention mechanism.
• Figure 4: A detailed overview of the feature pyramid. C2 to C5 represent multi-stage features from the view encoding network (ResNet18). "Upsample" denotes the upsamling operation, while "A" represents the addition operation.
• Figure 5: Schematic diagram of the data acquisition system for MAV action dataset.