Deformable ConvNets v2: More Deformable, Better Results

TL;DR

The paper introduces Deformable ConvNets v2 (DCNv2), which increases deformable modeling power by stacking deformable layers across more network stages and adding a modulation mechanism to adjust both sampling locations and their influence. It pairs these architectural changes with a training strategy that uses R-CNN feature mimicking to steer learned features toward object-focused representations, guided by a teacher network. Extensive COCO experiments show substantial improvements over DCNv1 and strong gains across Faster R-CNN and Mask R-CNN backbones, including when using stronger backbones. The results demonstrate that better focusing of deformable sampling and informative training signals yield significant gains in object detection and instance segmentation with modest computational overhead.

Abstract

The superior performance of Deformable Convolutional Networks arises from its ability to adapt to the geometric variations of objects. Through an examination of its adaptive behavior, we observe that while the spatial support for its neural features conforms more closely than regular ConvNets to object structure, this support may nevertheless extend well beyond the region of interest, causing features to be influenced by irrelevant image content. To address this problem, we present a reformulation of Deformable ConvNets that improves its ability to focus on pertinent image regions, through increased modeling power and stronger training. The modeling power is enhanced through a more comprehensive integration of deformable convolution within the network, and by introducing a modulation mechanism that expands the scope of deformation modeling. To effectively harness this enriched modeling capability, we guide network training via a proposed feature mimicking scheme that helps the network to learn features that reflect the object focus and classification power of R-CNN features. With the proposed contributions, this new version of Deformable ConvNets yields significant performance gains over the original model and produces leading results on the COCO benchmark for object detection and instance segmentation.

Figures (5)

• Figure 1: Spatial support of nodes in the last layer of the conv5 stage in a regular ConvNet, DCNv1 and DCNv2. The regular ConvNet baseline is Faster R-CNN + ResNet-50. In each subfigure, the effective sampling locations, effective receptive field, and error-bounded saliency regions are shown from the top to the bottom rows. Effective sampling locations are omitted in (c) as they are similar to those in (b), providing limited additional information. The visualized nodes (green points) are on a small object (left), a large object (middle), and the background (right).
• Figure 2: Spatial support of the 2 fc node in the per-RoI detection head, directly followed by the classification and the bounding box regression branches. Visualization is conducted on a regular ConvNet, DCNv1 and DCNv2. The regular ConvNet baseline is Faster R-CNN + ResNet-50. In each subfigure, the effective bin locations, effective receptive fields, and error-bounded saliency regions are shown from the top to the bottom rows, except for (c)$\sim$(e) where the effective bin locations are omitted as they provide little additional understanding over those in (a)$\sim$(b). The input RoIs (green boxes) are on a small object (left), a large object (middle), and the background (right).
• Figure 3: Network training with R-CNN feature mimicking.
• Figure 4: AP$^\text{bbox}$ scores of DCNv2 and regular ConvNets (Faster R-CNN + ResNet-50 / ResNet-101) on input images of varies resolution on the COCO 2017 test-dev set.
• Figure 5: Spatial support of nodes in the last layer of the conv5 stage in DCNv2 and regular ConvNets. Input images are of shorter side 400 pixels (left), 800 pixels (middle), and 1400 pixels (right), respectively. The effective receptive field and error-bounded saliency regions are shown in the top and bottom rows, respectively.