Foundation Model Priors Enhance Object Focus in Feature Space for Source-Free Object Detection

TL;DR

This paper addresses the challenge of source-free object detection under domain shift by enhancing object-focused representations rather than solely refining pseudo-labels. It introduces SPAR, which leverages offline foundation-model priors to regularize the detector's feature space toward foreground objects, and IRPL, an imbalance-aware, noise-robust pseudo-labeling loss that stabilizes learning under noisy supervision. The authors provide theory linking these components to tighter classification and localization bounds, and demonstrate consistent, architecture-agnostic gains across multiple SFOD benchmarks. The approach achieves competitive or state-of-the-art performance on several domain-shift scenarios, with ablations confirming the complementary benefits of SPAR and IRPL and improvements in long-tail categories. Overall, FALCON-SFOD offers a practical, plug-in enhancement to mean-teacher SFOD that leverages foundation priors for robust, object-centric adaptation.

Abstract

Current state-of-the-art approaches in Source-Free Object Detection (SFOD) typically rely on Mean-Teacher self-labeling. However, domain shift often reduces the detector's ability to maintain strong object-focused representations, causing high-confidence activations over background clutter. This weak object focus results in unreliable pseudo-labels from the detection head. While prior works mainly refine these pseudo-labels, they overlook the underlying need to strengthen the feature space itself. We propose FALCON-SFOD (Foundation-Aligned Learning with Clutter suppression and Noise robustness), a framework designed to enhance object-focused adaptation under domain shift. It consists of two complementary components. SPAR (Spatial Prior-Aware Regularization) leverages the generalization strength of vision foundation models to regularize the detector's feature space. Using class-agnostic binary masks derived from OV-SAM, SPAR promotes structured and foreground-focused activations by guiding the network toward object regions. IRPL (Imbalance-aware Noise Robust Pseudo-Labeling) complements SPAR by promoting balanced and noise-tolerant learning under severe foreground-background imbalance. Guided by a theoretical analysis that connects these designs to tighter localization and classification error bounds, FALCON-SFOD achieves competitive performance across SFOD benchmarks.

Figures (4)

• Figure 1: Towards Object-Focused Representations in SFOD. Two examples from the Foggy Cityscapes sakaridis2018semantic target set. In each example, the two columns show the student’s thresholded channel-mean feature maps from the last backbone layer (brighter = higher activation) and the model predictions, respectively. Existing state-of-the-art methods IRG irg and Simple-SFOD hao2024simplifying tend to produce less object-focused activations extending into background regions, leading to less precise object localization (red arrows) and false positives (orange arrows). In contrast, our method produces compact, object-shaped activations with tighter boxes and accurate labels, demonstrating stronger spatial coherence and object awareness. (Best viewed when zoomed in)
• Figure 2: Study of the impact of SPAR on source-free adaptation.
• Figure A.1: Performance on C $\rightarrow$ F with different m values used in the IRPL loss.
• Figure A.2: Additional qualitative results. Four examples from the Foggy Cityscapes sakaridis2018semantic target set. Mean-feature map is obtained from taking the channel-mean from the last layer of the student's backbone and thresholding at 0.6.Left: Baseline model hao2024simplifying produces spurious background activations, leading to missed detections or localization errors (red arrows) and false positives (orange arrows). Right: Our method effectively suppresses both feature-space confusion and class-label noise, resulting in clear activations and more accurate classification and object localization. Zoom in for best view.

Theorems & Definitions (8)

• Lemma 1
• Lemma 2
• Theorem 1
• Theorem 2
• proof
• proof
• proof
• proof