P2RBox: Point Prompt Oriented Object Detection with SAM

TL;DR

P2RBox addresses the cost of annotating oriented objects in aerial imagery by using single-point prompts with SAM to generate mask proposals, which are then refined via boundary-aware MIL and centrality-guided modules to select high-quality masks and predict rotation. An angle-prediction branch converts chosen masks into rotated bounding boxes, enabling pseudo-label training of classic detectors (RetinaNet, FCOS, Oriented R-CNN) and achieving substantial gains over prior point-supervised methods on DOTA v1.0 and DIOR-RBox. The framework combines a MIL-based semantic assessment with spatial guidance to reduce granularity ambiguity, providing a practical, parameter-light path to high-performance oriented detection with weak supervision. This approach broadens the applicability of point annotations in remote sensing by enabling effective training of strong detectors without fully supervised labels.

Abstract

Single-point annotation in oriented object detection of remote sensing scenarios is gaining increasing attention due to its cost-effectiveness. However, due to the granularity ambiguity of points, there is a significant performance gap between previous methods and those with fully supervision. In this study, we introduce P2RBox, which employs point prompt to generate rotated box (RBox) annotation for oriented object detection. P2RBox employs the SAM model to generate high-quality mask proposals. These proposals are then refined using the semantic and spatial information from annotation points. The best masks are converted into oriented boxes based on the feature directions suggested by the model. P2RBox incorporates two advanced guidance cues: Boundary Sensitive Mask guidance, which leverages semantic information, and Centrality guidance, which utilizes spatial information to reduce granularity ambiguity. This combination enhances detection capabilities significantly. To demonstrate the effectiveness of this method, enhancements based on the baseline were observed by integrating three different detectors. Furthermore, compared to the state-of-the-art point-annotated generative method PointOBB, P2RBox outperforms by about 29% mAP (62.43% vs 33.31%) on DOTA-v1.0 dataset, which provides possibilities for the practical application of point annotations.

Figures (8)

• Figure 1: Visual comparison of the highest confidence mask and its corresponding rotated box annotation generated by mask proposal generator (SAM) and P2RBox. The second row displays the results of the baseline method, while the Rotated boxes in the last row are generated by the SAE module. (Best viewed in color).
• Figure 2: The overview of training process of P2RBox, consisting of boundary-aware mask-guided MIL module, centrality-guided module. Initially, mask proposals are generated by a generator (SAM). Four point sets are constructed according the highest initial score mask to train boundary-aware mask-guided MIL moudle, which pursuing dataset-wide category consistency. The centrality-guided module generates offset penalty to enhance alignment between points and masks. The trained network will used to assesses mask quality, selecting the best proposals for detector training supervision. (Best viewed in color)
• Figure 3: Examples of detection results on the DOTA dataset using P2RBox (Oriented R-CNN).
• Figure 4: GT, minimum and SAE on category BD.
• Figure 5: Category BR, with mask proposals generated by SAM with annotated point and its circumscribed rbox.