DMTrack: Deformable State-Space Modeling for UAV Multi-Object Tracking with Kalman Fusion and Uncertainty-Aware Association

TL;DR

DMTrack targets UAV-based multi-object tracking under challenging non-linear motion and occlusions while discarding appearance cues. It combines DeformMamba, a deformable state-space predictor, with MotionGate, an adaptive Kalman-Mamba fusion module, and an uncertainty-aware matching strategy to maintain identities. The approach yields state-of-the-art MOTA and IDF1 on VisDrone-MOT and UAVDT without appearance models and runs in real time, demonstrating practical viability for UAV applications. This work highlights the value of hybrid, uncertainty-aware motion modeling for robust, appearance-free MOT in aerial settings.

Abstract

Multi-object tracking (MOT) from unmanned aerial vehicles (UAVs) presents unique challenges due to unpredictable object motion, frequent occlusions, and limited appearance cues inherent to aerial viewpoints. These issues are further exacerbated by abrupt UAV movements, leading to unreliable trajectory estimation and identity switches. Conventional motion models, such as Kalman filters or static sequence encoders, often fall short in capturing both linear and non-linear dynamics under such conditions. To tackle these limitations, we propose DMTrack, a deformable motion tracking framework tailored for UAV-based MOT. Our DMTrack introduces three key components: DeformMamba, a deformable state-space predictor that dynamically aggregates historical motion states for adaptive trajectory modeling; MotionGate, a lightweight gating module that fuses Kalman and Mamba predictions based on motion context and uncertainty; and an uncertainty-aware association strategy that enhances identity preservation by aligning motion trends with prediction confidence. Extensive experiments on the VisDrone-MOT and UAVDT benchmarks demonstrate that our DMTrack achieves state-of-the-art performance in identity consistency and tracking accuracy, particularly under high-speed and non-linear motion. Importantly, our method operates without appearance models and maintains competitive efficiency, highlighting its practicality for robust UAV-based tracking.

Figures (5)

• Figure 1: Under abrupt motion (yellow arrows), Kalman-only matching yields lower matching scores (0.6, 0.7), whereas DMTrack achieves higher confidence (0.8, 0.9) through deformable motion modeling, adaptive fusion, and uncertainty-aware association.
• Figure 2: Overview of our DMTrack framework. Our DMTrack first models motion using both Deformable Mamba and Kalman filters, then adaptively fuses their predictions via MotionGate. The final association relies solely on motion cues—IoU, trajectory trend, and uncertainty—to generate robust tracking results without appearance features.
• Figure 3: The DeformMamba module.The model predicts temporal offsets to interpolate informative keyframes, tokenizes them, and applies Mamba blocks for trajectory encoding. The final output is generated via a prediction head. Right: internal structure of the Mamba block.
• Figure 4: MotionGate module for adaptive motion fusion.computes per-dimension fusion weights from Kalman and Mamba inputs, generating a combined motion state and uncertainty. Left: gating network. Right: weighted fusion process.
• Figure 5: Visual comparison of our DMTrack with OC-SORT and MM-Track.Same numbers indicate consistent identities across frames. Our DMTrack shows better identity preservation than OC-SORT and MM-Track.