ReCon: Region-Controllable Data Augmentation with Rectification and Alignment for Object Detection

TL;DR

ReCon tackles the high cost of annotated data for object detection by enabling region-controllable augmentation within frozen diffusion generators. It introduces Region-Guided Rectification (RGR) and Region-Aligned Cross-Attention (RACA) to enforce spatial fidelity and semantic alignment during sampling, without retraining generators. Across COCO and VOC, and for multiple detectors, ReCon yields consistent improvements in mAP and data efficiency, outperforming traditional augmentation and several diffusion-based baselines, and proving effective in data-scarce and few-shot settings. The approach is plug-and-play with existing structure-controlled models and carries practical implications for scalable, high-fidelity synthetic data for detection tasks.

Abstract

The scale and quality of datasets are crucial for training robust perception models. However, obtaining large-scale annotated data is both costly and time-consuming. Generative models have emerged as a powerful tool for data augmentation by synthesizing samples that adhere to desired distributions. However, current generative approaches often rely on complex post-processing or extensive fine-tuning on massive datasets to achieve satisfactory results, and they remain prone to content-position mismatches and semantic leakage. To overcome these limitations, we introduce ReCon, a novel augmentation framework that enhances the capacity of structure-controllable generative models for object detection. ReCon integrates region-guided rectification into the diffusion sampling process, using feedback from a pre-trained perception model to rectify misgenerated regions within diffusion sampling process. We further propose region-aligned cross-attention to enforce spatial-semantic alignment between image regions and their textual cues, thereby improving both semantic consistency and overall image fidelity. Extensive experiments demonstrate that ReCon substantially improve the quality and trainability of generated data, achieving consistent performance gains across various datasets, backbone architectures, and data scales. Our code is available at https://github.com/haoweiz23/ReCon .

Figures (9)

• Figure 1: Overview of the ReCon Pipeline. ReCon enhances object-detection data generation by integrating region control into frozen, off-the-shelf models. It first compute the text embedding and instance masks, and then leverages a structural controllable diffusion model as the data generator and introduces region-guided rectification to refine generated results during the sampling process. Additionally, region-aligned cross-attention is incorporated to mitigate semantic leakage. Our method is plug-and-play and can be integrated with existing structure-controllable models.
• Figure 2: Sampling process with Region-Guided Rectification . We first identify incorrectly generated regions by IoU matching detection results with the original annotations using a grounding model. Next, we derive a rectification mask $\mathbf{M}$ and use $\mathbf{z}_t^{orig}$ sampled from the original data point to correct these errors.
• Figure 3: Pipeline of our Region-Aligned Cross-Attention. We first crop region-specific features from $\mathbf{z}_t^{in}$ using predefined regions $R_{gt}$. For regions belonging to the same category, we perform cross-attention with the corresponding text features. Finally, the interacted region image features are concatenated to produce $\mathbf{z}_t^{out}$.
• Figure 4: Data scaling on different COCO subsets.
• Figure 5: Visualization comparison of generated samples. Our methods show better image fidelity and content-annotation consistency.