GSAlign: Geometric and Semantic Alignment Network for Aerial-Ground Person Re-Identification

TL;DR

GSAlign tackles aerial-ground person re-identification by explicitly modeling geometric distortions and semantic misalignment through two modules: LTPS, a learnable thin-plate-spline transformer that warps features with rotation-aware control points, and DAM, which generates visibility-aware semantic masks to emphasize discriminative channels during training. Integrated into a ViT backbone with a view token, LTPS is applied progressively across layers, while DAM refines feature subspaces via dynamic prototype masking, leading to robust cross-view matching. On the CARGO dataset and real AG-ReID benchmarks, GSAlign achieves state-of-the-art results, notably improving cross-view pairs by large margins (e.g., +18.8% in mAP and +16.8% in Rank-1 over prior methods on aerial-ground settings), and ablations confirm the complementary benefits of LTPS and DAM. The work provides practical guidance on LTPS configuration (4 control points, full-layer insertion) and demonstrates that the added geometric-semantic alignment yields substantial gains with negligible inference cost, indicating strong potential for real-world UAV-to-ground surveillance systems.

Abstract

Aerial-Ground person re-identification (AG-ReID) is an emerging yet challenging task that aims to match pedestrian images captured from drastically different viewpoints, typically from unmanned aerial vehicles (UAVs) and ground-based surveillance cameras. The task poses significant challenges due to extreme viewpoint discrepancies, occlusions, and domain gaps between aerial and ground imagery. While prior works have made progress by learning cross-view representations, they remain limited in handling severe pose variations and spatial misalignment. To address these issues, we propose a Geometric and Semantic Alignment Network (GSAlign) tailored for AG-ReID. GSAlign introduces two key components to jointly tackle geometric distortion and semantic misalignment in aerial-ground matching: a Learnable Thin Plate Spline (LTPS) Module and a Dynamic Alignment Module (DAM). The LTPS module adaptively warps pedestrian features based on a set of learned keypoints, effectively compensating for geometric variations caused by extreme viewpoint changes. In parallel, the DAM estimates visibility-aware representation masks that highlight visible body regions at the semantic level, thereby alleviating the negative impact of occlusions and partial observations in cross-view correspondence. A comprehensive evaluation on CARGO with four matching protocols demonstrates the effectiveness of GSAlign, achieving significant improvements of +18.8\% in mAP and +16.8\% in Rank-1 accuracy over previous state-of-the-art methods on the aerial-ground setting. The code is available at: \textcolor{magenta}{https://github.com/stone96123/GSAlign}.

Figures (3)

• Figure 1: Illustration of the motivation of GSAlign.(a) Prior methods rely solely on implicit alignment, which is insufficient to fully address spatial and semantic distortions. (b) In contrast, our GSAlign performs explicit alignment at both the geometric and semantic levels via LTPS and visibility-aware semantic masks, respectively. This design equips GSAlign with a stronger capability for robust aerial-ground matching.
• Figure 2: Overview of the GSAlign architecture. Given aerial or ground-view inputs, GSAlign first applies an initial geometric transformation via a Learnable Thin Plate Spline (LTPS) module, followed by progressive alignment through LTPS blocks inserted before each ViT layer. In parallel, a Dynamic Alignment Module (DAM) generates a visibility-aware semantic mask according to the input image, which is then applied to the representations of other images in the batch to suppress irrelevant or occluded features.
• Figure 3: Qualitative comparison before and after LTPS alignment. The input image (red) exhibits significant geometric distortion due to extreme viewpoint variation. After applying the Learnable Thin Plate Spline (LTPS) transformation (green), the image is spatially rectified, highlighting improved geometric consistency and local structure alignment.