Towards Unbiased Source-Free Object Detection via Vision Foundation Models

TL;DR

Source-Free Object Detection (SFOD) suffers from source bias under data-privacy constraints. This work presents DSOD, a Vision Foundation Model (VFM)–assisted SFOD framework that injects VFM semantics via Unbiased Feature Injection and enforces regularization through Semantic-aware Feature Regularization, with Domain-aware Adaptive Weighting stabilizing training; a Dual-Teacher Distillation path yields a VFM-free variant (DSOD-Distill) for practical deployment. Empirical results show DSOD achieving state-of-the-art performance across Normal-to-Foggy, Cross-scene, and Synthetic-to-Real shifts (e.g., 48.1% AP, 39.3% AP, and 61.4% AP, respectively), and DSOD-Distill providing competitive accuracy with reduced inference burden. The approach demonstrates that VFMs can provide complementary, robust representations to reduce source bias in SFOD while enabling scalable, resource-efficient deployment. Overall, DSOD offers a principled framework for incorporating foundation-model knowledge into privacy-preserving domain adaptation for dense vision tasks.

Abstract

Source-Free Object Detection (SFOD) has garnered much attention in recent years by eliminating the need of source-domain data in cross-domain tasks, but existing SFOD methods suffer from the Source Bias problem, i.e. the adapted model remains skewed towards the source domain, leading to poor generalization and error accumulation during self-training. To overcome this challenge, we propose Debiased Source-free Object Detection (DSOD), a novel VFM-assisted SFOD framework that can effectively mitigate source bias with the help of powerful VFMs. Specifically, we propose Unified Feature Injection (UFI) module that integrates VFM features into the CNN backbone through Simple-Scale Extension (SSE) and Domain-aware Adaptive Weighting (DAAW). Then, we propose Semantic-aware Feature Regularization (SAFR) that constrains feature learning to prevent overfitting to source domain characteristics. Furthermore, we propose a VFM-free variant, termed DSOD-distill for computation-restricted scenarios through a novel Dual-Teacher distillation scheme. Extensive experiments on multiple benchmarks demonstrate that DSOD outperforms state-of-the-art SFOD methods, achieving 48.1% AP on Normal-to-Foggy weather adaptation, 39.3% AP on Cross-scene adaptation, and 61.4% AP on Synthetic-to-Real adaptation.

Figures (8)

• Figure 1: Conceptual comparison of different domain adaptation paradigms. (I) Ideal Adaptation: Achieves complete knowledge transfer from the source domain (red) to the target domain (blue). (II) Traditional SFOD: Prone to source bias, as indicated by the predominance of red circles representing source-domain characteristics. (III) UDAOD: Enables more effective adaptation but requires access to source-domain data. (IV) VFM-assisted SFOD: Mitigates source bias by incorporating unbiased knowledge (purple) while eliminating the need for source-domain data access.
• Figure 2: Overview of our method DSOD and DSOD-Distill. In the Stage I, we train a hybrid model (DSOD) on target-domain data with the assistance of VFM. Then, in Stage II, we transfer the knowledge from DSOD to a VFM-free variant, DSOD-Distill through Dual-Teacher distillation.
• Figure 3: The main components (student-part) of DSOD. To utilize the generalizable and semantic-rich features, we operate in two aspect: (I) Unbiased Feature Injection (UFI) and (II) Semantic-Aware Feature Regularization (SAFR). Also, we propose a Domain-Aware Adaptive Weighting (DAAW) strategy to select proper weighting and stablize the training.
• Figure 4: Illustration of the orthogonal feature space. The fused feature vector $\textbf{F}_{\text{fuse}}$, obtained through addition, preserves the information from both original $\textbf{F}_{\text{CNN}}$ and $\textbf{F}_{\text{DINO}}$.
• Figure 5: Visualization of prediction instability under different fusion weights. Red circle highlights the regions where objects are missing compared to the non-fusion baseline model ($w$=0).