Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On-the-Fly OVD Adaptation with FLAME: Few-shot Localization via Active Marginal-Samples Exploration

Yehonathan Refael, Amit Aides, Aviad Barzilai, George Leifman, Genady Beryozkin, Vered Silverman, Bolous Jaber, Tomer Shekel

TL;DR

This work tackles the challenge of adapting open-vocabulary remote-sensing detectors to user-defined targets with minimal labeling and no full-model fine-tuning. It introduces FLAME, a one-step active learning strategy that retrieves margin-focused marginal samples via density estimation and diversity clustering, then trains a lightweight classifier on a handful of user-labeled examples atop a frozen zero-shot OVD. Theoretical support shows that a classifier can be determined by its margin, justifying the few-shot, margin-driven retraining, with proofs in the Appendix. Empirically, FLAME yields state-of-the-art FSOD performance on DOTA and DIOR with CPU-only adaptation times around one minute per label, offering a practical, cost-efficient path to user-specific RS object detection.

Abstract

Open-vocabulary object detection (OVD) models offer remarkable flexibility by detecting objects from arbitrary text queries. However, their zero-shot performance in specialized domains like Remote Sensing (RS) is often compromised by the inherent ambiguity of natural language, limiting critical downstream applications. For instance, an OVD model may struggle to distinguish between fine-grained classes such as "fishing boat" and "yacht" since their embeddings are similar and often inseparable. This can hamper specific user goals, such as monitoring illegal fishing, by producing irrelevant detections. To address this, we propose a cascaded approach that couples the broad generalization of a large pre-trained OVD model with a lightweight few-shot classifier. Our method first employs the zero-shot model to generate high-recall object proposals. These proposals are then refined for high precision by a compact classifier trained in real-time on only a handful of user-annotated examples - drastically reducing the high costs of RS imagery annotation.The core of our framework is FLAME, a one-step active learning strategy that selects the most informative samples for training. FLAME identifies, on the fly, uncertain marginal candidates near the decision boundary using density estimation, followed by clustering to ensure sample diversity. This efficient sampling technique achieves high accuracy without costly full-model fine-tuning and enables instant adaptation, within less then a minute, which is significantly faster than state-of-the-art alternatives.Our method consistently surpasses state-of-the-art performance on RS benchmarks, establishing a practical and resource-efficient framework for adapting foundation models to specific user needs.

On-the-Fly OVD Adaptation with FLAME: Few-shot Localization via Active Marginal-Samples Exploration

TL;DR

This work tackles the challenge of adapting open-vocabulary remote-sensing detectors to user-defined targets with minimal labeling and no full-model fine-tuning. It introduces FLAME, a one-step active learning strategy that retrieves margin-focused marginal samples via density estimation and diversity clustering, then trains a lightweight classifier on a handful of user-labeled examples atop a frozen zero-shot OVD. Theoretical support shows that a classifier can be determined by its margin, justifying the few-shot, margin-driven retraining, with proofs in the Appendix. Empirically, FLAME yields state-of-the-art FSOD performance on DOTA and DIOR with CPU-only adaptation times around one minute per label, offering a practical, cost-efficient path to user-specific RS object detection.

Abstract

Open-vocabulary object detection (OVD) models offer remarkable flexibility by detecting objects from arbitrary text queries. However, their zero-shot performance in specialized domains like Remote Sensing (RS) is often compromised by the inherent ambiguity of natural language, limiting critical downstream applications. For instance, an OVD model may struggle to distinguish between fine-grained classes such as "fishing boat" and "yacht" since their embeddings are similar and often inseparable. This can hamper specific user goals, such as monitoring illegal fishing, by producing irrelevant detections. To address this, we propose a cascaded approach that couples the broad generalization of a large pre-trained OVD model with a lightweight few-shot classifier. Our method first employs the zero-shot model to generate high-recall object proposals. These proposals are then refined for high precision by a compact classifier trained in real-time on only a handful of user-annotated examples - drastically reducing the high costs of RS imagery annotation.The core of our framework is FLAME, a one-step active learning strategy that selects the most informative samples for training. FLAME identifies, on the fly, uncertain marginal candidates near the decision boundary using density estimation, followed by clustering to ensure sample diversity. This efficient sampling technique achieves high accuracy without costly full-model fine-tuning and enables instant adaptation, within less then a minute, which is significantly faster than state-of-the-art alternatives.Our method consistently surpasses state-of-the-art performance on RS benchmarks, establishing a practical and resource-efficient framework for adapting foundation models to specific user needs.
Paper Structure (7 sections, 3 theorems, 10 equations, 2 figures, 2 tables)

This paper contains 7 sections, 3 theorems, 10 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Theory and Motivations.
  4. Marginal Samples Retrieval.
  5. Experiments
  6. Discussion
  7. Proofs

Key Result

Lemma 2.1

Let $\{(x_i,y_i)\}_{i=1}^n$ be linearly separable with $y_i\in\{\pm1\}$. Consider the hard–margin SVM Let $(w^\star,b^\star)$ be an optimal solution and define the support set $S \;:=\; \bigl\{ i\in[n] : y_i\,(w^{\star\top}x_i+b^\star)=1 \bigr\}.$ Then, Consequently, eq:svmP and eq:svmPS have the same optimal solutions. In particular, retraining the hard–margin SVM after removing all non–support

Figures (2)

  • Figure 1: A visual demonstration of performance improvement from Zero-Shot to Few-Shot detection using DIOR dataset dior2023rs. The Zero-Shot model (center) produces noisy and unreliable results, identifying the 'chimneys' but with low confidence and accompanied by several false positives. Our Few-Shot method (right) refines this output, successfully eliminating the false positives and accurately detecting all four chimneys shown in the Ground Truth (left). Similarly, the bottom row showcases the detection of a 'dam'. The Zero-Shot model struggles with false positives such as 'bridge' and 'harbor', which are corrected by the more precise Few-Shot approach.
  • Figure 2: Overview of the proposed few-shot sampling method. The method follows the stages: (1) uncertainty-based filtering using density estimation to identify ambiguous candidates near the decision boundary, (2) clustering-based diversity sampling to ensure representative coverage, (3) interactive user annotation of the selected samples, (4) conditional data augmentation with SMOTE or SVM-SMOTE to balance classes, and (5) lightweight classifier training (e.g., SVM or MLP) on the augmented set. This cascaded process refines the zero-shot proposals from a large open-vocabulary detector into an accurate, real-time few-shot classifier without full-model fine-tuning

Theorems & Definitions (7)

  • Lemma 2.1: Support--determination for hard–margin SVM
  • Remark 2.2: Kernel SVM
  • Lemma 2.2: Support-determination for soft-margin SVM
  • Lemma 2.3: Support examples--determination for homogeneous networks
  • proof : Proof of Lemma \ref{['lem::margin_svm']}
  • proof : Proof of Lemma \ref{['lem::soft_margin_svm']}
  • proof : Proof of Lemma \ref{['lem::margin_homogeneous_networks']}