Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Frequency: Scoring-Driven Debiasing for Object Detection via Blueprint-Prompted Image Synthesis

Xinhao Cai, Liulei Li, Gensheng Pei, Tao Chen, Jinshan Pan, Yazhou Yao, Wenguan Wang

TL;DR

This paper tackles dataset biases in object detection by moving beyond frequency-based debiasing to a representation-focused framework. It introduces Representation Score (RS) to diagnose representational gaps, a dynamic debiasing engine to recalibrate layouts, and a visual blueprint–prompted L2I synthesis paired with a generative alignment mechanism to ensure geometry-faithful data. The approach yields substantial gains for underrepresented object groups and achieves state-of-the-art fidelity in synthetic data generation, with a $15.9$ mAP improvement in layout accuracy and notable mAP gains across rare, border, large, and small objects. The work demonstrates that bias-aware data generation, guided by accurate diagnostics and geometry-preserving prompts, can significantly enhance robustness and generalization in real-world detectors.

Abstract

This paper presents a generation-based debiasing framework for object detection. Prior debiasing methods are often limited by the representation diversity of samples, while naive generative augmentation often preserves the biases it aims to solve. Moreover, our analysis reveals that simply generating more data for rare classes is suboptimal due to two core issues: i) instance frequency is an incomplete proxy for the true data needs of a model, and ii) current layout-to-image synthesis lacks the fidelity and control to generate high-quality, complex scenes. To overcome this, we introduce the representation score (RS) to diagnose representational gaps beyond mere frequency, guiding the creation of new, unbiased layouts. To ensure high-quality synthesis, we replace ambiguous text prompts with a precise visual blueprint and employ a generative alignment strategy, which fosters communication between the detector and generator. Our method significantly narrows the performance gap for underrepresented object groups, \eg, improving large/rare instances by 4.4/3.6 mAP over the baseline, and surpassing prior L2I synthesis models by 15.9 mAP for layout accuracy in generated images.

Beyond Frequency: Scoring-Driven Debiasing for Object Detection via Blueprint-Prompted Image Synthesis

TL;DR

This paper tackles dataset biases in object detection by moving beyond frequency-based debiasing to a representation-focused framework. It introduces Representation Score (RS) to diagnose representational gaps, a dynamic debiasing engine to recalibrate layouts, and a visual blueprint–prompted L2I synthesis paired with a generative alignment mechanism to ensure geometry-faithful data. The approach yields substantial gains for underrepresented object groups and achieves state-of-the-art fidelity in synthetic data generation, with a mAP improvement in layout accuracy and notable mAP gains across rare, border, large, and small objects. The work demonstrates that bias-aware data generation, guided by accurate diagnostics and geometry-preserving prompts, can significantly enhance robustness and generalization in real-world detectors.

Abstract

This paper presents a generation-based debiasing framework for object detection. Prior debiasing methods are often limited by the representation diversity of samples, while naive generative augmentation often preserves the biases it aims to solve. Moreover, our analysis reveals that simply generating more data for rare classes is suboptimal due to two core issues: i) instance frequency is an incomplete proxy for the true data needs of a model, and ii) current layout-to-image synthesis lacks the fidelity and control to generate high-quality, complex scenes. To overcome this, we introduce the representation score (RS) to diagnose representational gaps beyond mere frequency, guiding the creation of new, unbiased layouts. To ensure high-quality synthesis, we replace ambiguous text prompts with a precise visual blueprint and employ a generative alignment strategy, which fosters communication between the detector and generator. Our method significantly narrows the performance gap for underrepresented object groups, \eg, improving large/rare instances by 4.4/3.6 mAP over the baseline, and surpassing prior L2I synthesis models by 15.9 mAP for layout accuracy in generated images.
Paper Structure (16 sections, 14 equations, 15 figures, 5 tables)

This paper contains 16 sections, 14 equations, 15 figures, 5 tables.

Table of Contents

  1. Introduction
  2. The Frequency Trap and Fidelity Gap: A Motivating Study
  3. Visual-Prompted Dynamic Debiasing for Object Detection
  4. Dynamic Debiasing via Scoring-Driven Layout Generation
  5. High-Fidelity L2I Synthesis with Visual Blueprints
  6. Related Work
  7. Experiment
  8. Experimental Results
  9. Diagnostic Experiments
  10. Conclusion
  11. Appendix
  12. Use of Large Language Models (LLMs)
  13. Metric Definition
  14. Experimental Setup
  15. Additional Analysis
  16. ...and 1 more sections

Figures (15)

  • Figure 1: Comparison of four data enrichment strategies with respect to object location, category frequency, and object size as the dataset scale increases from 1/4 by factors of 4/3, 2, and 4.
  • Figure 2: Illustration for layout recalibration.
  • Figure 3: Illustration for blueprint construction.
  • Figure 4: Visualization of recalibrated layouts, showing objects with updated sizes and positions, and new instances (left). Our method can generate geometry-faithful images compared to prior SOTA (right).
  • Figure 5: Ablative studies of essential components in our proposed method on MS COCO 2017 lin2014microsoft.
  • ...and 10 more figures