Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Control Copy-Paste: Controllable Diffusion-Based Augmentation Method for Remote Sensing Few-Shot Object Detection

Yanxing Liu, Jiancheng Pan, Bingchen Zhang

TL;DR

Limited annotated data in optical remote sensing FSOD leads to overfitting. The paper identifies that both object and contextual diversity influence performance and that decoupling contexts from instances is essential. It introduces Control Copy-Paste, a controllable diffusion-based pipeline that injects few-shot objects into diverse contexts with orientation alignment and diffusion conditioning. On the DIOR dataset, the method yields substantial improvements, underscoring the importance of contextual diversity for RSIs.

Abstract

Few-shot object detection (FSOD) for optical remote sensing images aims to detect rare objects with only a few annotated bounding boxes. The limited training data makes it difficult to represent the data distribution of realistic remote sensing scenes, which results in the notorious overfitting problem. Current researchers have begun to enhance the diversity of few-shot novel instances by leveraging diffusion models to solve the overfitting problem. However, naively increasing the diversity of objects is insufficient, as surrounding contexts also play a crucial role in object detection, and in cases where the object diversity is sufficient, the detector tends to overfit to monotonous contexts. Accordingly, we propose Control Copy-Paste, a controllable diffusion-based method to enhance the performance of FSOD by leveraging diverse contextual information. Specifically, we seamlessly inject a few-shot novel objects into images with diverse contexts by a conditional diffusion model. We also develop an orientation alignment strategy to mitigate the integration distortion caused by varying aspect ratios of instances. Experiments on the public DIOR dataset demonstrate that our method can improve detection performance by an average of 10.76%.

Control Copy-Paste: Controllable Diffusion-Based Augmentation Method for Remote Sensing Few-Shot Object Detection

TL;DR

Limited annotated data in optical remote sensing FSOD leads to overfitting. The paper identifies that both object and contextual diversity influence performance and that decoupling contexts from instances is essential. It introduces Control Copy-Paste, a controllable diffusion-based pipeline that injects few-shot objects into diverse contexts with orientation alignment and diffusion conditioning. On the DIOR dataset, the method yields substantial improvements, underscoring the importance of contextual diversity for RSIs.

Abstract

Few-shot object detection (FSOD) for optical remote sensing images aims to detect rare objects with only a few annotated bounding boxes. The limited training data makes it difficult to represent the data distribution of realistic remote sensing scenes, which results in the notorious overfitting problem. Current researchers have begun to enhance the diversity of few-shot novel instances by leveraging diffusion models to solve the overfitting problem. However, naively increasing the diversity of objects is insufficient, as surrounding contexts also play a crucial role in object detection, and in cases where the object diversity is sufficient, the detector tends to overfit to monotonous contexts. Accordingly, we propose Control Copy-Paste, a controllable diffusion-based method to enhance the performance of FSOD by leveraging diverse contextual information. Specifically, we seamlessly inject a few-shot novel objects into images with diverse contexts by a conditional diffusion model. We also develop an orientation alignment strategy to mitigate the integration distortion caused by varying aspect ratios of instances. Experiments on the public DIOR dataset demonstrate that our method can improve detection performance by an average of 10.76%.

Paper Structure

This paper contains 9 sections, 3 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Preliminary
  4. What limits the performance of FSOD: An Analysis of Factors Influencing FSOD
  5. Control Copy-Paste Pipeline
  6. Results
  7. Datasets and Implementation details
  8. Result Analysis
  9. Discussion

Figures (3)

  • Figure 1: The illustration of the proposed Control Copy-Paste pipeline.
  • Figure 2: Analysis of different components. (a). Detection performance varies across different instances. (b). Detection performance varies across different contexts.
  • Figure 3: The architecture framework of the proposed pipeline for few-shot object detection on remote sensing images. The detector is first trained on a base dataset containing abundant objects to learn domain-agnostic knowledge of RSIs. Few-shot novel objects are then injected into multiple context images through a class-agnostic conditional diffusion model. Finally, the synthetic dataset is combined with the few-shot dataset to fine-tune the detectors.