Point2RBox-v3: Self-Bootstrapping from Point Annotations via Integrated Pseudo-Label Refinement and Utilization

TL;DR

Point2RBox-v3 tackles weakly-supervised oriented object detection from point annotations by introducing two core innovations: Progressive Label Assignment (PLA), which dynamically assigns ground-truth points to appropriate FPN levels using evolving pseudo labels, and Prior-Guided Dynamic Mask Loss (PGDM-Loss), which smartly routes mask generation between Watershed and SAM and leverages class-aware priors. The combination yields higher-quality pseudo labels and better utilization of multi-scale features, achieving state-of-the-art performance across multiple remote-sensing datasets in both end-to-end and two-stage settings, and proving effective in partial weakly-supervised scenarios. The work highlights the importance of dynamic label assignment and hybrid mask supervision for robust, scalable weakly-supervised OOD. Overall, Point2RBox-v3 advances the practicality and performance of point-supervised oriented object detection, with broad implications for real-world deployments where detailed annotations are costly.

Abstract

Driven by the growing need for Oriented Object Detection (OOD), learning from point annotations under a weakly-supervised framework has emerged as a promising alternative to costly and laborious manual labeling. In this paper, we discuss two deficiencies in existing point-supervised methods: inefficient utilization and poor quality of pseudo labels. Therefore, we present Point2RBox-v3. At the core are two principles: 1) Progressive Label Assignment (PLA). It dynamically estimates instance sizes in a coarse yet intelligent manner at different stages of the training process, enabling the use of label assignment methods. 2) Prior-Guided Dynamic Mask Loss (PGDM-Loss). It is an enhancement of the Voronoi Watershed Loss from Point2RBox-v2, which overcomes the shortcomings of Watershed in its poor performance in sparse scenes and SAM's poor performance in dense scenes. To our knowledge, Point2RBox-v3 is the first model to employ dynamic pseudo labels for label assignment, and it creatively complements the advantages of SAM model with the watershed algorithm, which achieves excellent performance in both sparse and dense scenes. Our solution gives competitive performance, especially in scenarios with large variations in object size or sparse object occurrences: 66.09%/56.86%/41.28%/46.40%/19.60%/45.96% on DOTA-v1.0/DOTA-v1.5/DOTA-v2.0/DIOR/STAR/RSAR.

Figures (5)

• Figure 1: (Left) Visual comparisons with the state-of-the-art method Point2RBox-v2. The boxes detected by our method wrap the objects more tightly, with fewer missed detections. (Right) Radar plot comparing the performance of our method with 10 other state-of-the-art methods across 6 benchmark datasets. Methods with '(e)' in the legend indicate end-to-end version, while those without '(e)' represent two-stage version.
• Figure 2: The training pipeline of Point2RBox-v3. Progressive Label Assign utilizes scale info from pseudo label to dynamically assign gt point (see Figure \ref{['fig:pla']}). Prior-Guided Dynamic Mask provides enhanced mask supervision information (see Figure \ref{['fig:sam_watershed_comparison']}). $\mathcal{L}_{others}$ are the loss functions inherited from Point2RBox-v2 (see Appendix \ref{['sec:loss_from_v2']}).
• Figure 3: The process of Progressive Label Assignment (PLA). Points of different colors represent those assigned to different feature pyramid levels P2, P3, P4, P5, P6 for label assignment. As training progresses, the label assignment strategy evolves. It begins with using fixed Watershed regions in the early stages and transitions to leveraging dynamic, network-generated dimensions in the middle-to-late phases. This evolution guides ground truth points to be assigned to more suitable FPN levels over time.
• Figure 4: Comparison between watershed and SAM masks on DOTA-v1.0. The red patches with yellow edges represent the masks generated by the model. The processing result in the top-right corner shows significant over-segmentation by SAM, which causes the masks to visually merge into a large, incorrect region.
• Figure 5: Efficacy of Class-Specific Watershed on Overlapping Instances. The top row shows challenging cases with overlapping objects. The standard Watershed algorithm produces unsatisfactory masks under these conditions (second row). Our class-specific Watershed (third and fourth rows) effectively mitigates this issue, leading to a noticeable improvement in mask separation and quality.