Structure Tensor Representation for Robust Oriented Object Detection

TL;DR

Extensive evaluations demonstrate that the proposed structure tensor representation outperforms previous methods in both fully-supervised and weakly supervised tasks, achieving high precision in angular prediction with minimal computational overhead.

Abstract

Oriented object detection predicts orientation in addition to object location and bounding box. Precisely predicting orientation remains challenging due to angular periodicity, which introduces boundary discontinuity issues and symmetry ambiguities. Inspired by classical works on edge and corner detection, this paper proposes to represent orientation in oriented bounding boxes as a structure tensor. This representation combines the strengths of Gaussian-based methods and angle-coder solutions, providing a simple yet efficient approach that is robust to angular periodicity issues without additional hyperparameters. Extensive evaluations across five datasets demonstrate that the proposed structure tensor representation outperforms previous methods in both fully-supervised and weakly supervised tasks, achieving high precision in angular prediction with minimal computational overhead. Thus, this work establishes structure tensors as a robust and modular alternative for encoding orientation in oriented object detection. We make our code publicly available, allowing for seamless integration into existing object detectors.

Figures (4)

• Figure 1: The proposed angle representation. Comparison between the traditional oriented bounding box format $(x, y, w, h, \theta)$ (in blue) and our structure tensor representation $T$ (in orange). Orientation and anisotropy are represented in T by its eigenvalues $\lambda_1$ and $\lambda_2$, and their corresponding eigenvectors $v_1, v_2$.
• Figure 2: Structure tensor representation in a neural network. (a) During training, the backbone extracts image features $f$, which are used for classification and regression. The angle head predicts orientation as a structure tensor $T_{pred}$, and the ground truth $OBB_{gt}$ is encoded into $T_{gt}$ for angle loss computation. (b) At inference, $T_{pred}$ is decoded into the standard OBB format $(x,y,w,h,\theta)$. Blue denotes standard detector components, while green highlights our method.
• Figure 3: Loss behavior with isotropic structure tensors. Rectangular objects exhibit $\pi$ periodicity, with loss increasing as angle error grows, while isotropic (square-like) objects yield consistently low loss regardless of angle error. Introducing anisotropy results in a loss with $\frac{\pi}{2}$ periodicity that yields higher values as prediction error increases.
• Figure 4: Qualitative results of the proposed approach on several datasets. On the top, from left to right, detection examples from HRSC2016, DOTA, and ICDAR2015 are shown. On the bottom, the left image corresponds to the MSRA-TD500 dataset, while the one on the right belongs to the C19TD test dataset.