A Closer Look at Data Augmentation Strategies for Finetuning-Based Low/Few-Shot Object Detection

TL;DR

This paper investigates energy-efficient data augmentation strategies for finetuning-based low- and few-shot object detection using a lightweight detector. It compares handcrafted and automated augmentation methods (AutoAugment, RandAugment, AugMix) across three industrial datasets, measuring both $AP_{50}$ and energy use with CodeCarbon, and introduces the Efficiency Factor $EF = \frac{AP_{50}}{1+EC}$ to capture performance-energy trade-offs. The study shows that energy costs often offset marginal $AP_{50}$ gains and highlights the need for energy-efficient augmentation designs in data-scarce regimes, supported by extensive ablations between custom and automated DAs under FSL and LSL. These findings guide practical deployment on edge devices and motivate future work in green AI for robust, energy-conscious vision systems.

Abstract

Current methods for low- and few-shot object detection have primarily focused on enhancing model performance for detecting objects. One common approach to achieve this is by combining model finetuning with data augmentation strategies. However, little attention has been given to the energy efficiency of these approaches in data-scarce regimes. This paper seeks to conduct a comprehensive empirical study that examines both model performance and energy efficiency of custom data augmentations and automated data augmentation selection strategies when combined with a lightweight object detector. The methods are evaluated in three different benchmark datasets in terms of their performance and energy consumption, and the Efficiency Factor is employed to gain insights into their effectiveness considering both performance and efficiency. Consequently, it is shown that in many cases, the performance gains of data augmentation strategies are overshadowed by their increased energy usage, necessitating the development of more energy efficient data augmentation strategies to address data scarcity.

Figures (3)

• Figure 1: $AP_{50}$ with respect to energy consumption for different DA strategies and numbers of shots.
• Figure 2: Energy consumption difference between different DA strategies and vanilla finetuning for a varying number of shots.
• Figure 3: $AP_{50}$ performance difference between finetuning-based approaches and Base.