Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging Transformer Decoder for Automotive Radar Object Detection

Changxu Zhang, Zhaoze Wang, Tai Fei, Christopher Grimm, Yi Jin, Claas Tebruegge, Ernst Warsitz, Markus Gardill

TL;DR

This work addresses 3D automotive radar object detection by replacing traditional CNN heads with a Transformer Decoder that performs end-to-end set prediction. It introduces Pyramid Token Fusion (PTF) to unify multi-scale radar features into a single token memory, enabling global context modeling across Doppler, azimuth, and range dimensions via learnable object queries. The method employs Hungarian matching with a combined L_Bbox and L_Class loss to produce NMS-free predictions, achieving state-of-the-art results on the RADDet dataset and demonstrating robust localization under challenging radar conditions. The approach highlights the viability and practical impact of transformer-based radar perception for streamlined, end-to-end perception pipelines in autonomous driving.

Abstract

In this paper, we present a Transformer-based architecture for 3D radar object detection that uses a novel Transformer Decoder as the prediction head to directly regress 3D bounding boxes and class scores from radar feature representations. To bridge multi-scale radar features and the decoder, we propose Pyramid Token Fusion (PTF), a lightweight module that converts a feature pyramid into a unified, scale-aware token sequence. By formulating detection as a set prediction problem with learnable object queries and positional encodings, our design models long-range spatial-temporal correlations and cross-feature interactions. This approach eliminates dense proposal generation and heuristic post-processing such as extensive non-maximum suppression (NMS) tuning. We evaluate the proposed framework on the RADDet, where it achieves significant improvements over state-of-the-art radar-only baselines.

Leveraging Transformer Decoder for Automotive Radar Object Detection

TL;DR

This work addresses 3D automotive radar object detection by replacing traditional CNN heads with a Transformer Decoder that performs end-to-end set prediction. It introduces Pyramid Token Fusion (PTF) to unify multi-scale radar features into a single token memory, enabling global context modeling across Doppler, azimuth, and range dimensions via learnable object queries. The method employs Hungarian matching with a combined L_Bbox and L_Class loss to produce NMS-free predictions, achieving state-of-the-art results on the RADDet dataset and demonstrating robust localization under challenging radar conditions. The approach highlights the viability and practical impact of transformer-based radar perception for streamlined, end-to-end perception pipelines in autonomous driving.

Abstract

In this paper, we present a Transformer-based architecture for 3D radar object detection that uses a novel Transformer Decoder as the prediction head to directly regress 3D bounding boxes and class scores from radar feature representations. To bridge multi-scale radar features and the decoder, we propose Pyramid Token Fusion (PTF), a lightweight module that converts a feature pyramid into a unified, scale-aware token sequence. By formulating detection as a set prediction problem with learnable object queries and positional encodings, our design models long-range spatial-temporal correlations and cross-feature interactions. This approach eliminates dense proposal generation and heuristic post-processing such as extensive non-maximum suppression (NMS) tuning. We evaluate the proposed framework on the RADDet, where it achieves significant improvements over state-of-the-art radar-only baselines.
Paper Structure (16 sections, 12 equations, 3 figures, 2 tables)

This paper contains 16 sections, 12 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. CNN-based Radar Object Detection
  4. DETR-based Object Detection
  5. Transformer-based Radar Object Detection
  6. Methodology
  7. Pyramid Token Fusion
  8. Transformer Decoder Head
  9. Loss Function
  10. Experiments
  11. Dataset
  12. Evaluation Metrics
  13. Training Details
  14. Benchmark Comparison and Performance Evaluation
  15. Qualitative Result Analysis
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Overall architecture of the proposed model. The RAD cube is encoded by a hierarchical transformer backbone to produce multi-scale features, which are augmented with positional and level embeddings in the PTF module and fused into a unified token sequence. This sequence serves as memory for the transformer decoder, whose object queries generate class scores and 3D bounding box predictions via two lightweight heads.
  • Figure 2: Examples of detection results from our model shown for five different scenes (a-e). The top row displays the camera images. The middle row shows our model's predictions, overlaid on the RA map (left, x: azimuth, y: range) and RD map (right, x: Doppler, y: range). The bottom row presents the corresponding ground-truth annotations. Bounding box colors indicate object classes: red for cars, purple for pedestrians, and blue for cyclists.
  • Figure 3: Cross-attention maps on multi-scale feature maps (right), shown alongside RA (left) map with predicted bounding boxes. The spatial dimensions of the attention maps correspond to those of the RA map.