MTSGL: Multi-Task Structure Guided Learning for Robust and Interpretable SAR Aircraft Recognition

TL;DR

This work tackles robust and interpretable aircraft recognition in SAR imagery by introducing a structure-based annotation method and a multi-task structure guided learning (MTSGL) framework. MTSGL integrates two structure-focused tasks—Structural Semantic Awareness (SSA) and Structural Consistency Regularization (SCR)—with a Pareto-oriented gradient descent to balance learning across tasks, enabling the encoder to internalize aircraft structural knowledge. On the MT-SARD dataset, MTSGL achieves state-of-the-art performance, with a notable OA improvement from 83.37% (baseline) to 90.21% and a Kappa rise to approximately 0.886 when using Pareto optimization, while also providing interpretable structural evidence via segmentation and attention visualizations. The approach demonstrates enhanced robustness, particularly under limited training data and annotation imperfections, and offers a practical path toward more interpretable SAR ATR systems, with future work oriented toward weakly supervised annotation to further reduce labeling costs.

Abstract

Aircraft recognition in synthetic aperture radar (SAR) imagery is a fundamental mission in both military and civilian applications. Recently deep learning (DL) has emerged a dominant paradigm for its explosive performance on extracting discriminative features. However, current classification algorithms focus primarily on learning decision hyperplane without enough comprehension on aircraft structural knowledge. Inspired by the fined aircraft annotation methods for optical remote sensing images (RSI), we first introduce a structure-based SAR aircraft annotations approach to provide structural and compositional supplement information. On this basis, we propose a multi-task structure guided learning (MTSGL) network for robust and interpretable SAR aircraft recognition. Besides the classification task, MTSGL includes a structural semantic awareness (SSA) module and a structural consistency regularization (SCR) module. The SSA is designed to capture structure semantic information, which is conducive to gain human-like comprehension of aircraft knowledge. The SCR helps maintain the geometric consistency between the aircraft structure in SAR imagery and the proposed annotation. In this process, the structural attribute can be disentangled in a geometrically meaningful manner. In conclusion, the MTSGL is presented with the expert-level aircraft prior knowledge and structure guided learning paradigm, aiming to comprehend the aircraft concept in a way analogous to the human cognitive process. Extensive experiments are conducted on a self-constructed multi-task SAR aircraft recognition dataset (MT-SARD) and the effective results illustrate the superiority of robustness and interpretation ability of the proposed MTSGL.

Figures (18)

• Figure 1: Illustration of existing aircraft annotation methods in the optical and SAR imagery. (a) Optical image. (b) Landmark-based RN1583. (c) Polygon-based RN1580. (d) Wing-based RN1582. (e) SAR image. (f) Posture-based RN1585. (g) Head and tail-based RN1591. (h) Structure-based (Proposed).
• Figure 2: Overall description of the proposed template-based annotation for SAR aircraft target. (a) Predefined binary templates and scale information of aircraft with lengths in blue blocks and wingspans in green blocks. (b) Initial annotation based on horizontal bounding boxes in panoramic images. (c) Expert-level annotation on target area. This process consists of two manual steps and one automated step, including interpretation, alignment and matching. (d) The formed SAR aircraft recognition dataset, involving origin images, categories, masks and projections.
• Figure 3: Overall illustration of the proposed MTSGL. The multi-objectives are obtained by passing the input images through one shared feature encoder and three task-specific modules. Consequently, the encoder's gradient is obtained by merging three task-specific gradients backpropagated from their respective losses. At testing phase, only the classification branch is preserved to predict the class scores.
• Figure 4: The transformer-based structure of the MTSGL. (a). The forward process given the input image and the corresponding template during the training stage. Note that the SCR branch (yellow) are discarded at inference stage. The feature encoder (blue) and the SSA branch (green) adopt the U-Net structure. (b). The U-Net detailed structure. (c). The Swin Transformer block.
• Figure 5: Illustration of the distance map from the segmentation ground truth. (a) SAR image. (b) Ground Truth. (c) Distance Map.