Performance-guided Reinforced Active Learning for Object Detection

TL;DR

MGRAL introduces a performance-guided reinforcement learning approach to active learning for object detection by tying batch selection to $ ext{mAP}$ improvements through a policy-gradient sampler. It tackles non-differentiable, combinatorial batch selection by using $igtriangleup ext{mAP}$ as the reward and a semi-supervised detector proxy to estimate gains, while a lookup-table accelerator enables scalable training. The method demonstrates superior or competitive $ ext{mAP}$ gains on VOC and COCO across backbones, and analyses reveal that downstream performance signals drive effective and diverse sample selection. This framework provides a practical, RL-driven paradigm for data-efficient object detection, with potential extensions to faster mAP estimation and online prediction-based RL alternatives.

Abstract

Active learning (AL) strategies aim to train high-performance models with minimal labeling efforts, only selecting the most informative instances for annotation. Current approaches to evaluating data informativeness predominantly focus on the data's distribution or intrinsic information content and do not directly correlate with downstream task performance, such as mean average precision (mAP) in object detection. Thus, we propose Performance-guided (i.e. mAP-guided) Reinforced Active Learning for Object Detection (MGRAL), a novel approach that leverages the concept of expected model output changes as informativeness. To address the combinatorial explosion challenge of batch sample selection and the non-differentiable correlation between model performance and selected batches, MGRAL skillfully employs a reinforcement learning-based sampling agent that optimizes selection using policy gradient with mAP improvement as reward. Moreover, to reduce the computational overhead of mAP estimation with unlabeled samples, MGRAL utilizes an unsupervised way with fast look-up tables, ensuring feasible deployment. We evaluate MGRAL's active learning performance on detection tasks over PASCAL VOC and COCO benchmarks. Our approach demonstrates the highest AL curve with convincing visualizations, establishing a new paradigm in reinforcement learning-driven active object detection.

Figures (4)

• Figure 1: Overview of Performance-guided (mAP-guided) Reinforced Active Learning (MGRAL) for object detection. The RL-based agent learns to select most informative samples directly using performance gains ($\Delta \text{mAP}$) as reward.
• Figure 2: Data sampling agent architecture.
• Figure 3: Comparative performance of active learning methods. The mAP is plotted against the number of labeled images.
• Figure 4: Visualizations of the selected samples with highest scores by different AL methods during first cycle on VOC.