PointOBB-v3: Expanding Performance Boundaries of Single Point-Supervised Oriented Object Detection

TL;DR

PointOBB-v3 tackles the challenge of single point-supervised oriented object detection by introducing a three-view learning framework that jointly optimizes object scale and orientation without extra priors. It combines a Scale Augmentation Module (SSC and SSFF) with a Symmetry-inspired angle module (Dense/Dense and Dense/Sparse matching plus SSA loss), and offers an end-to-end variant with an Instance-Aware Weighting strategy to stabilize joint training. Across DIOR-R, DOTA, FAIR1M, STAR, and RSAR, the method achieves a mean improvement of about 3.56% in mAP50 over prior SOTA, with notable gains in both pseudo-label generation and end-to-end efficiency. This work provides a practical, scalable approach to accurate rotated bounding boxes under weak supervision, enabling broader application to aerial imagery and beyond.

Abstract

With the growing demand for oriented object detection (OOD), recent studies on point-supervised OOD have attracted significant interest. In this paper, we propose PointOBB-v3, a stronger single point-supervised OOD framework. Compared to existing methods, it generates pseudo rotated boxes without additional priors and incorporates support for the end-to-end paradigm. PointOBB-v3 functions by integrating three unique image views: the original view, a resized view, and a rotated/flipped (rot/flp) view. Based on the views, a scale augmentation module and an angle acquisition module are constructed. In the first module, a Scale-Sensitive Consistency (SSC) loss and a Scale-Sensitive Feature Fusion (SSFF) module are introduced to improve the model's ability to estimate object scale. To achieve precise angle predictions, the second module employs symmetry-based self-supervised learning. Additionally, we introduce an end-to-end version that eliminates the pseudo-label generation process by integrating a detector branch and introduces an Instance-Aware Weighting (IAW) strategy to focus on high-quality predictions. We conducted extensive experiments on the DIOR-R, DOTA-v1.0/v1.5/v2.0, FAIR1M, STAR, and RSAR datasets. Across all these datasets, our method achieves an average improvement in accuracy of 3.56% in comparison to previous state-of-the-art methods. The code will be available at https://github.com/ZpyWHU/PointOBB-v3.

Figures (7)

• Figure 1: The main paradigmatic types of existing OOD. Compared to RBox and HBox labels, point labels have lower costs and higher efficiency. "Prior" indicates using additional human knowledge priors. "E2E" indicates training in an end-to-end manner, and "mAP" shows performance on the DOTA-v1.0 dataset using mAP$_{50}$ metric.
• Figure 2: The pipeline of the PointOBB-v3. PointOBB-v3 consists of three distinct views, which serve as the foundation for constructing an angle acquisition module and a scale augmentation module. The latter integrates a Scale-Sensitive Consistency (SSC) loss and a Scale-Sensitive Feature Fusion (SSFF) module to improve the model's capability in perceiving scale variations. The angle acquisition module incorporates a Dense-to-Dense sample assignment mechanism to facilitate angle learning, optimized through the Self-Supervised Angle (SSA) loss. Moreover, a Dense-to-Sparse (DS) matching strategy is introduced to achieve more accurate object angle estimation. During training, a progressive multi-view switching strategy is implemented, enabling the transition between resized views, rotated/flipped views, and their associated modules.
• Figure 3: Line graphs of instance score distributions from the original and the resized views before and after employing the proposed SSC loss, the distributions are grouped by ratios.
• Figure 4: The Pipeline of End-to-end version of PointOBB-v3. Building on the original MIL-based framework, a detection branch is integrated. This newly added branch shares the backbone and neck parameters with the MIL head, promoting parameter efficiency. To facilitate the joint training of both branches, the outputs of the MIL branch and the detection branch are aligned through the calculation of L$_{e2e}$. Furthermore, an Instance-Aware Weighting (IAW) strategy is proposed to further enhance the integration and collaborative training of the two branches.
• Figure 5: The visual detection results. Compared existing methods include: PointOBB (2024) luo2024pointobb, PointOBB-v2 (2024) ren2024pointobbv2simplerfasterstronger, and Point2RBox (2024) yu2024point2rbox. The last row showcases the results of the proposed PointOBB-v3 combined with Oriented R-CNN.