Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ATRNet-STAR: A Large Dataset and Benchmark Towards Remote Sensing Object Recognition in the Wild

Yongxiang Liu, Weijie Li, Li Liu, Jie Zhou, Bowen Peng, Yafei Song, Xuying Xiong, Wei Yang, Tianpeng Liu, Zhen Liu, Xiang Li

TL;DR

ATRNet-STAR addresses the critical need for large-scale, diverse SAR ATR data by introducing a 194,324-sample dataset of 40 vehicle types collected under 5 scenes with X- and Ku-band quad-polarization, plus two data formats and rich metadata. The authors establish ATRBench, a 7-setting, 15-method benchmark for both classification and detection, to enable robust, reproducible comparisons and to probe performance under realistic open-world conditions. Extensive experiments reveal that imaging angle, scene complexity, and data format (magnitude vs. complex) significantly affect recognition, with modern architectures and SAR foundation-models providing the strongest gains under SOC but facing challenges in complex EOC scenarios. The work also demonstrates clear benefits of domain-specific pretraining and transfer learning, and it opens avenues for few-shot, incremental, and generative approaches, ultimately aiming to catalyze progress in SAR ATR research and practical open-world deployment.

Abstract

The absence of publicly available, large-scale, high-quality datasets for Synthetic Aperture Radar Automatic Target Recognition (SAR ATR) has significantly hindered the application of rapidly advancing deep learning techniques, which hold huge potential to unlock new capabilities in this field. This is primarily because collecting large volumes of diverse target samples from SAR images is prohibitively expensive, largely due to privacy concerns, the characteristics of microwave radar imagery perception, and the need for specialized expertise in data annotation. Throughout the history of SAR ATR research, there have been only a number of small datasets, mainly including targets like ships, airplanes, buildings, etc. There is only one vehicle dataset MSTAR collected in the 1990s, which has been a valuable source for SAR ATR. To fill this gap, this paper introduces a large-scale, new dataset named ATRNet-STAR with 40 different vehicle categories collected under various realistic imaging conditions and scenes. It marks a substantial advancement in dataset scale and diversity, comprising over 190,000 well-annotated samples, 10 times larger than its predecessor, the famous MSTAR. Building such a large dataset is a challenging task, and the data collection scheme will be detailed. Secondly, we illustrate the value of ATRNet-STAR via extensively evaluating the performance of 15 representative methods with 7 different experimental settings on challenging classification and detection benchmarks derived from the dataset. Finally, based on our extensive experiments, we identify valuable insights for SAR ATR and discuss potential future research directions in this field. We hope that the scale, diversity, and benchmark of ATRNet-STAR can significantly facilitate the advancement of SAR ATR.

ATRNet-STAR: A Large Dataset and Benchmark Towards Remote Sensing Object Recognition in the Wild

TL;DR

ATRNet-STAR addresses the critical need for large-scale, diverse SAR ATR data by introducing a 194,324-sample dataset of 40 vehicle types collected under 5 scenes with X- and Ku-band quad-polarization, plus two data formats and rich metadata. The authors establish ATRBench, a 7-setting, 15-method benchmark for both classification and detection, to enable robust, reproducible comparisons and to probe performance under realistic open-world conditions. Extensive experiments reveal that imaging angle, scene complexity, and data format (magnitude vs. complex) significantly affect recognition, with modern architectures and SAR foundation-models providing the strongest gains under SOC but facing challenges in complex EOC scenarios. The work also demonstrates clear benefits of domain-specific pretraining and transfer learning, and it opens avenues for few-shot, incremental, and generative approaches, ultimately aiming to catalyze progress in SAR ATR research and practical open-world deployment.

Abstract

The absence of publicly available, large-scale, high-quality datasets for Synthetic Aperture Radar Automatic Target Recognition (SAR ATR) has significantly hindered the application of rapidly advancing deep learning techniques, which hold huge potential to unlock new capabilities in this field. This is primarily because collecting large volumes of diverse target samples from SAR images is prohibitively expensive, largely due to privacy concerns, the characteristics of microwave radar imagery perception, and the need for specialized expertise in data annotation. Throughout the history of SAR ATR research, there have been only a number of small datasets, mainly including targets like ships, airplanes, buildings, etc. There is only one vehicle dataset MSTAR collected in the 1990s, which has been a valuable source for SAR ATR. To fill this gap, this paper introduces a large-scale, new dataset named ATRNet-STAR with 40 different vehicle categories collected under various realistic imaging conditions and scenes. It marks a substantial advancement in dataset scale and diversity, comprising over 190,000 well-annotated samples, 10 times larger than its predecessor, the famous MSTAR. Building such a large dataset is a challenging task, and the data collection scheme will be detailed. Secondly, we illustrate the value of ATRNet-STAR via extensively evaluating the performance of 15 representative methods with 7 different experimental settings on challenging classification and detection benchmarks derived from the dataset. Finally, based on our extensive experiments, we identify valuable insights for SAR ATR and discuss potential future research directions in this field. We hope that the scale, diversity, and benchmark of ATRNet-STAR can significantly facilitate the advancement of SAR ATR.
Paper Structure (15 sections, 12 figures, 6 tables)

This paper contains 15 sections, 12 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Datasets for SAR target classification
  4. Review of SAR Target Classification Algorithms
  5. Building ATRNet-STAR: Why, What, How
  6. Construct program
  7. Data acquisition
  8. Professional system
  9. Statistical Analysis
  10. Dataset value
  11. Benchmarks
  12. Experimental settings
  13. Results and analyses
  14. Discussion
  15. Conclusion

Figures (12)

  • Figure 1: Our ATRNet-STAR dataset contains 40 distinct target types, collected with the aim of replacing the outdated though widely used MSTAR dataset and making a significant contribution to the advancement of SAR ATR research.
  • Figure 2: Motivation of our ATRNet-STAR.Subfigure (a) depicts the most frequent keywords in 21,780 journal papers published in remote sensing (TGRS, JSTARS, GRSL, ISPRS Journal, and JAG) from 2020 to 2024. The size of each word is proportional to its frequency, highlighting that concepts such as Synthetic Aperture Radar (SAR), image classification, and object detection have garnered substantial attention. Subfigure (b) focuses on the number of publications related to SAR Automatic Target Recognition (ATR) over the past five years, a cross-area of the concepts highlighted in Subfigure (a). As the pioneering dataset for SAR target classification, MSTAR has long served as the predominant benchmark due to its unique data diversity and accumulated benchmarks.
  • Figure 3: Importance of Dataset. Taking the MSTAR dataset as an example, subfigures (a1)-(a3) describe the common data inputs of existing algorithms. The MSTAR dataset provides processed magnitude data, raw complex data, and corresponding metadata. Subfigures (b1)-(b2) show its mainstream experimental settings that are based on various imaging conditions of the MSTAR dataset. Other experimental settings include target configuration/version variation and simulation settings (simulation noise erosion and occlusion) 10283916. However, the limited MSTAR acquisition conditions restrict the further study of the SAR ATR. For example, the current occlusion usually uses zero values to fill some image regions, but the real occlusion in our dataset subfigure (b3) is not the same as the existing simulation. (pseudo-color for better visualization of SAR magnitude images, and subfigures (b) are cropped and centered for better visualization of target signature variations.)
  • Figure 4: Illustrations of data acquisition. We annotate and cut to build target slices with corresponding metadata information. The range dimension of slant range complex data is in the line of sight direction, which results in the deformation of the target shape in this dimension. Therefore, we also provide ground range images and slant range amplitude data.
  • Figure 5: Taxonomic systems. For civilian vehicles, the taxonomy is based on Chinese and European vehicle classification standards GA802－2019Passengercarclassification, according to the vehicle's purpose, structure, size, and mass. For military vehicles, we followed the MSTAR taxonomic system Ross1999SAR. (a) Taxonomic systems of ATRNet-STAR. Our dataset is a comprehensive civilian vehicle map covering 4 classes, 21 subclasses, and 40 types. We provide a detailed illustration of the histogram distribution for these 40 vehicle types. (b) Target classes of SAR vehicle datasets. We statistically analyze the number of civilian and military vehicle classes and types in SAR vehicle datasets. (c) List of vehicle abbreviations.
  • ...and 7 more figures