Fourier Angle Alignment for Oriented Object Detection in Remote Sensing

TL;DR

Fourier Angle Alignment is introduced, which analyses angle information through frequency spectrum and aligns the main direction to a certain orientation, and is proposed two plug and play modules : FAAFusion and FAA Head.

Abstract

In remote sensing rotated object detection, mainstream methods suffer from two bottlenecks, directional incoherence at detector neck and task conflict at detecting head. Ulitising fourier rotation equivariance, we introduce Fourier Angle Alignment, which analyses angle information through frequency spectrum and aligns the main direction to a certain orientation. Then we propose two plug and play modules : FAAFusion and FAA Head. FAAFusion works at the detector neck, aligning the main direction of higher-level features to the lower-level features and then fusing them. FAA Head serves as a new detection head, which pre-aligns RoI features to a canonical angle and adds them to the original features before classification and regression. Experiments on DOTA-v1.0, DOTA-v1.5 and HRSC2016 show that our method can greatly improve previous work. Particularly, our method achieves new state-of-the-art results of 78.72% mAP on DOTA-v1.0 and 72.28% mAP on DOTA-v1.5 datasets with single scale training and testing, validating the efficacy of our approach in remote sensing object detection. The code is made publicly available at https://github.com/gcy0423/Fourier-Angle-Alignment .

Figures (6)

• Figure 1: The position of FAAFusion and FAA Head.
• Figure 2: Directional incoherence. High-level feature (c) has strong semantics and large receptive fields but captures the coarse orientation. For example, from (c) we can only infer that the object lies horizontally. Yet many down-samplings blur spatial details, so the orientation signal is low-frequency and vague. Low-level feature (b) keeps rich edges, corners, and textures, giving clear and high-frequency orientation cues.
• Figure 3: Task conflict in the detection head. As the orientation varies, the angle predictions are expected to be different while the class predictions are expected to be the same.
• Figure 4: The top row shows the target images in the spatial domain. The second row gives their matching frequency spectra. In the bottom row, red lines point out the main energy directions in each spectrum. (a) If the spatial picture is rotated, its spectrum distribution will be rotated as well. (b) The main spectral direction of one rectangle-like object is perpendicular to the direction of its major axis.
• Figure 5: Structures of FAA, FAAFusion and FAA Head. For FAA, it can be divided into two steps. First, we estimate the main direction in frequent domain. And then FAA accepts an external angle and aligns the main direction with it. For FAAFusion, we use the main direction of the lower local feature as the external angle and align the higher local feature with it. For FAA Head, it aligns the main direction of RoI features with $0\degree$ to obtain the rotation invariant features.