RQFormer: Rotated Query Transformer for End-to-End Oriented Object Detection

TL;DR

RQFormer tackles two core challenges in end-to-end oriented object detection: misalignment between positional queries and keys and the proliferation of similar queries that hinder one-to-one label assignment. It introduces Rotated RoI Attention to align oriented region features with queries and Selective Distinct Queries to harvest high-quality, distinct queries from intermediate decoder layers, reducing redundancy without extra auxiliary structures. The method achieves strong results across remote sensing and text benchmarks, outperforming a range of CNN- and Transformer-based baselines, and demonstrates good generalization to horizontal detection tasks. With a parameter-efficient design and competitive inference speed, RQFormer offers a practical end-to-end solution for oriented object detection and points to future enhancements through contextual integration and further efficiency gains.

Abstract

Oriented object detection presents a challenging task due to the presence of object instances with multiple orientations, varying scales, and dense distributions. Recently, end-to-end detectors have made significant strides by employing attention mechanisms and refining a fixed number of queries through consecutive decoder layers. However, existing end-to-end oriented object detectors still face two primary challenges: 1) misalignment between positional queries and keys, leading to inconsistency between classification and localization; and 2) the presence of a large number of similar queries, which complicates one-to-one label assignments and optimization. To address these limitations, we propose an end-to-end oriented detector called the Rotated Query Transformer, which integrates two key technologies: Rotated RoI Attention (RRoI Attention) and Selective Distinct Queries (SDQ). First, RRoI Attention aligns positional queries and keys from oriented regions of interest through cross-attention. Second, SDQ collects queries from intermediate decoder layers and filters out similar ones to generate distinct queries, thereby facilitating the optimization of one-to-one label assignments. Finally, extensive experiments conducted on four remote sensing datasets and one scene text dataset demonstrate the effectiveness of our method. To further validate its generalization capability, we also extend our approach to horizontal object detection The code is available at \url{https://github.com/wokaikaixinxin/RQFormer}.

Figures (12)

• Figure 1: (a) Object instances in oriented object detection exhibit arbitrary orientation, different scales and dense distribution. (b) Visualization of RRoI attention without using CAM. RRoI attention align features and mostly attends to object extremities such as the front and back of ships, planes and vehicles.
• Figure 2: Overall framework. Decoder incorporates Rotated RoI attention (RRoI attention) and Selective Distinct Queries (SDQ). RRoI attention serves as cross-attention, aligning keys with positional queries. SDQ collects queries from intermediate decoder layers and filters similar queries.
• Figure 3: (a) Decoder layer with Rotated RoI attention. (b) Rotated RoI attention.
• Figure 4: (a) Basic process for queries update layer by layer in decoder. (b) Selective Distinct Queries.
• Figure 5: The ratios of similar queries under different high IoU threshold on DIOR-R. Compared with baseline, SDQ significantly filters similar queries.