Improving Object Detection via Local-global Contrastive Learning

TL;DR

This work presents a novel image-to-image translation method that optimises the appearance of object instances through spatial attention masks, implicitly delineating the scene into foreground regions associated with the target object instances and background non-object regions.

Abstract

Visual domain gaps often impact object detection performance. Image-to-image translation can mitigate this effect, where contrastive approaches enable learning of the image-to-image mapping under unsupervised regimes. However, existing methods often fail to handle content-rich scenes with multiple object instances, which manifests in unsatisfactory detection performance. Sensitivity to such instance-level content is typically only gained through object annotations, which can be expensive to obtain. Towards addressing this issue, we present a novel image-to-image translation method that specifically targets cross-domain object detection. We formulate our approach as a contrastive learning framework with an inductive prior that optimises the appearance of object instances through spatial attention masks, implicitly delineating the scene into foreground regions associated with the target object instances and background non-object regions. Instead of relying on object annotations to explicitly account for object instances during translation, our approach learns to represent objects by contrasting local-global information. This affords investigation of an under-explored challenge: obtaining performant detection, under domain shifts, without relying on object annotations nor detector model fine-tuning. We experiment with multiple cross-domain object detection settings across three challenging benchmarks and report state-of-the-art performance. Project page: https://local-global-detection.github.io

Figures (10)

• Figure 1: Left: visual domains, unseen during object detector training, hurt detection performance. Middle: global image-to-image translation (foggy $\rightarrow$ clear weather) provides some benefit to downstream detection performance, yet homogeneous image translation strategies result in small objects, with low contrast regions, that remain undetectable. Right: Our local-global approach is guided to better delineate objects during translation and thus cross-domain detection is improved.
• Figure 2: Overview of the proposed method - see text for further details.
• Figure 3: Col $1$: input images. Cols $2-4$: learned foreground attention masks. Local-global self-supervision accentuates semantic object content regions and improves translation in areas critical for object detection (e.g. people, cars). Col $5$: translated output images.
• Figure 4: t-SNE feature visuliazation; we randomly sample object features corresponding to salient objects (red) and image background regions (blue).
• Figure S1: Visualization of the learned foreground attention masks of the proposed supervised model (rows 1-4) and self-supervised local-global model (rows 5-8). Column 1 shows the input (foggy weather) image, columns 2--4 visualize attention masks from 3 different channels $A_k$ and column 5 shows the translated (clean weather) result.