Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Learning via Memory: Retrieval-Augmented Detector Adaptation

Yanan Jian, Fuxun Yu, Qi Zhang, William Levine, Brandon Dubbs, Nikolaos Karianakis

TL;DR

A novel way of online adapting any off-the-shelf object detection model to a novel domain without retraining the detector model is presented, which could significantly outperform baselines in adapting a detector to novel domains.

Abstract

This paper presents a novel way of online adapting any off-the-shelf object detection model to a novel domain without retraining the detector model. Inspired by how humans quickly learn knowledge of a new subject (e.g., memorization), we allow the detector to look up similar object concepts from memory during test time. This is achieved through a retrieval augmented classification (RAC) module together with a memory bank that can be flexibly updated with new domain knowledge. We experimented with various off-the-shelf open-set detector and close-set detectors. With only a tiny memory bank (e.g., 10 images per category) and being training-free, our online learning method could significantly outperform baselines in adapting a detector to novel domains.

Online Learning via Memory: Retrieval-Augmented Detector Adaptation

TL;DR

A novel way of online adapting any off-the-shelf object detection model to a novel domain without retraining the detector model is presented, which could significantly outperform baselines in adapting a detector to novel domains.

Abstract

This paper presents a novel way of online adapting any off-the-shelf object detection model to a novel domain without retraining the detector model. Inspired by how humans quickly learn knowledge of a new subject (e.g., memorization), we allow the detector to look up similar object concepts from memory during test time. This is achieved through a retrieval augmented classification (RAC) module together with a memory bank that can be flexibly updated with new domain knowledge. We experimented with various off-the-shelf open-set detector and close-set detectors. With only a tiny memory bank (e.g., 10 images per category) and being training-free, our online learning method could significantly outperform baselines in adapting a detector to novel domains.
Paper Structure (14 sections, 4 figures, 5 tables)

This paper contains 14 sections, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Retrieval-Augmented Detector Adaptation
  3. Pre-requisite: Object Proposal Model
  4. Memory Bank
  5. Retrieval-Augmented (RAC) Modules
  6. Experiments
  7. Experiment Setup
  8. Proposal Network Evaluation
  9. Memory Bank Ablation
  10. Memory Bank Sampling Method
  11. Memory Bank Size Comparison
  12. RAC Backbone Ablation: Pre-trained vs Finetuning
  13. Visual Analysis for RAC Process
  14. Conclusion

Figures (4)

  • Figure 1: Comparison between (a) traditional offline detector learning vs. (b) proposed online learning paradigm. With an updatable memory bank and a retrieval-augmented (RAC) module, we could leverage any off-the-shelf frozen detector to quickly detect new domain concepts by retrieving them from memory bank.
  • Figure 2: Context-Aware RAC Workflow.
  • Figure 3: Mean AP vs Memory Bank Size Ablation.
  • Figure 4: Visual Analysis for the RAC Process.