Adaptive Image Zoom-in with Bounding Box Transformation for UAV Object Detection

TL;DR

This work tackles the challenge of detecting small objects in UAV imagery by proposing ZoomDet, a non-uniform image zooming framework that adaptively magnifies object regions via an offset-based mapping learned by a lightweight OffsetNet. It introduces a novel object zooming loss to train the mapping and a corner-aligned bounding box transformation to align annotations between the original and zoomed spaces, enabling end-to-end training and inference with standard detectors. Across VisDrone, UAVDT, and SeaDronesSee, ZoomDet yields substantial improvements in mAP, notably on SeaDronesSee where Faster R-CNN gains over 8 points with only a few milliseconds of extra latency, and it remains architecture-agnostic, compatible with both two-stage and one-stage detectors. The approach demonstrates strong potential for aerial perception tasks and can extend to vision-language models, offering a flexible, efficient path to better small-object detection in remote sensing contexts.

Abstract

Detecting objects from UAV-captured images is challenging due to the small object size. In this work, a simple and efficient adaptive zoom-in framework is explored for object detection on UAV images. The main motivation is that the foreground objects are generally smaller and sparser than those in common scene images, which hinders the optimization of effective object detectors. We thus aim to zoom in adaptively on the objects to better capture object features for the detection task. To achieve the goal, two core designs are required: \textcolor{black}{i) How to conduct non-uniform zooming on each image efficiently? ii) How to enable object detection training and inference with the zoomed image space?} Correspondingly, a lightweight offset prediction scheme coupled with a novel box-based zooming objective is introduced to learn non-uniform zooming on the input image. Based on the learned zooming transformation, a corner-aligned bounding box transformation method is proposed. The method warps the ground-truth bounding boxes to the zoomed space to learn object detection, and warps the predicted bounding boxes back to the original space during inference. We conduct extensive experiments on three representative UAV object detection datasets, including VisDrone, UAVDT, and SeaDronesSee. The proposed ZoomDet is architecture-independent and can be applied to an arbitrary object detection architecture. Remarkably, on the SeaDronesSee dataset, ZoomDet offers more than 8.4 absolute gain of mAP with a Faster R-CNN model, with only about 3 ms additional latency. The code is available at https://github.com/twangnh/zoomdet_code.

Figures (9)

• Figure 1: Illustration of Motivation. Unlike prior patch-based methods that generate patch candidates and perform object detection on the patches, the goal is to directly perform detection on an adaptively zoomed image.
• Figure 2: Illustration of forward bounding box label transformation (green arrow) and backward prediction transformation (blue arrow).
• Figure 3: Overall framework of our proposed Method. Comparing the baseline approach that uniformly scales the image and performs detection, a non-uniform zoom-in is conducted to magnify the objects of interest and thus help better detect them; the detected bounding boxes are transformed back to obtain the final detection results. Note that the original input image may be large in resolution; thus, the baseline object detection pipeline involves a downsampling step to reduce resolution.
• Figure 4: Ablation study on hyper-parameters $\alpha$ and $\beta$. The top two figures show the effect of these parameters on the zooming loss shape. The bottom figure shows the ablation results by independently varying each parameter with the other fixed. The results are obtained on VisDrone dataset.
• Figure 5: Empirical validation of error induced with forward and backward bounding box transformation. The top row shows example GT bounding boxes on the original image, the second row shows the transformed GT bounding boxes on the zoomed image, the last row shows the GT bounding boxes transformed back to the original space. The bar chart shows the IoU between the original and the backward-transformed boxes.