Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ABRA: Teleporting Fine-Tuned Knowledge Across Domains for Open-Vocabulary Object Detection

Mattia Bernardi, Chiara Cappellino, Matteo Mosconi, Enver Sangineto, Angelo Porrello, Simone Calderara

Abstract

Although recent Open-Vocabulary Object Detection architectures, such as Grounding DINO, demonstrate strong zero-shot capabilities, their performance degrades significantly under domain shifts. Moreover, many domains of practical interest, such as nighttime or foggy scenes, lack large annotated datasets, preventing direct fine-tuning. In this paper, we introduce Aligned Basis Relocation for Adaptation(ABRA), a method that transfers class-specific detection knowledge from a labeled source domain to a target domain where no training images containing these classes are accessible. ABRA formulates this adaptation as a geometric transport problem in the weight space of a pretrained detector, aligning source and target domain experts to transport class-specific knowledge. Extensive experiments across challenging domain shifts demonstrate that ABRA successfully teleports class-level specialization under multiple adverse conditions. Our code will be made public upon acceptance.

ABRA: Teleporting Fine-Tuned Knowledge Across Domains for Open-Vocabulary Object Detection

Abstract

Although recent Open-Vocabulary Object Detection architectures, such as Grounding DINO, demonstrate strong zero-shot capabilities, their performance degrades significantly under domain shifts. Moreover, many domains of practical interest, such as nighttime or foggy scenes, lack large annotated datasets, preventing direct fine-tuning. In this paper, we introduce Aligned Basis Relocation for Adaptation(ABRA), a method that transfers class-specific detection knowledge from a labeled source domain to a target domain where no training images containing these classes are accessible. ABRA formulates this adaptation as a geometric transport problem in the weight space of a pretrained detector, aligning source and target domain experts to transport class-specific knowledge. Extensive experiments across challenging domain shifts demonstrate that ABRA successfully teleports class-level specialization under multiple adverse conditions. Our code will be made public upon acceptance.
Paper Structure (15 sections, 19 equations, 5 figures, 5 tables, 1 algorithm)

This paper contains 15 sections, 19 equations, 5 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Aligned Basis Relocation for Adaptation
  5. Domain Expert through Objectification
  6. Class Experts through Singular Vectors
  7. Teleportation through Weight Alignment
  8. Experiments
  9. Benchmarking
  10. Main Results
  11. Additional Analysis
  12. Conclusions
  13. Orthogonal Alignment of Singular Subspaces
  14. Implementation Details
  15. Qualitative Results

Figures (5)

  • Figure 1: Goal. Transfer class-specific knowledge from the source domain to unavailable target classes without access to target-domain data.
  • Figure 2: Overview of the transfer pipeline. Starting from domain experts obtained through Objectification, class-specific residuals learned via SVFT in the source domain are analytically transported to unseen target domains.
  • Figure 3: Objectification. Each domain is fine-tuned using a unified "object" label to capture domain appearance.
  • Figure 4: Effect of specialization on Cityscapes. Dedicating one expert per class (ABRA) yields higher AP$_{50}$ than using a single unified expert (Merge).
  • Figure 5: Few-shot transfer evaluation. Cityscapes comparison (mAP) between the standard pretrained backbone ($\theta_0$) and ABRA across varying numbers of shots.