LEARN: A Unified Framework for Multi-Task Domain Adapt Few-Shot Learning

TL;DR

The paper introduces LEARN, a unified, modular framework that integrates domain adaptation with few-shot learning across image classification, object detection, and video classification. It features an iterative $n$-shot protocol and optional self-supervised pre-training, controlled via Hydra/config files to enable flexible, reproducible experiments and easy extension to many-shot settings. The authors describe representative algorithms for each task, provide a detailed experiment protocol, and benchmark across multiple domain-shift datasets, demonstrating competitive few-shot performance and scalable evaluation. Open-source code accompanies the framework, highlighting its practicality for researchers and developers to run domain-adapted few-shot learning across multiple CV tasks.

Abstract

Both few-shot learning and domain adaptation sub-fields in Computer Vision have seen significant recent progress in terms of the availability of state-of-the-art algorithms and datasets. Frameworks have been developed for each sub-field; however, building a common system or framework that combines both is something that has not been explored. As part of our research, we present the first unified framework that combines domain adaptation for the few-shot learning setting across 3 different tasks - image classification, object detection and video classification. Our framework is highly modular with the capability to support few-shot learning with/without the inclusion of domain adaptation depending on the algorithm. Furthermore, the most important configurable feature of our framework is the on-the-fly setup for incremental $n$-shot tasks with the optional capability to configure the system to scale to a traditional many-shot task. With more focus on Self-Supervised Learning (SSL) for current few-shot learning approaches, our system also supports multiple SSL pre-training configurations. To test our framework's capabilities, we provide benchmarks on a wide range of algorithms and datasets across different task and problem settings. The code is open source has been made publicly available here: https://gitlab.kitware.com/darpa_learn/learn

Figures (7)

• Figure 1: The LEARN Framework Concept Figure showing the different components of our Domain Adapt Few-Shot learning workflow. The Few-Shot domain adaptation block shows a sample images from DomainNet peng2019moment ClipArt and Sketch for the class "Aircraft Carrier".
• Figure 2: Task workflow across the different tasks - IC: Image Classification, OD: Object Detection and VC: Video Classification from the LEARN framework. In addition to the multitask few-shot learning, the framework supports custom domain adapt scenarios as mentioned in the diagram along with the capability to setup SSL pre-training using unlabeled data.
• Figure 3: Overview of the image domains across the $3$ image classification datasets used in our benchmarking. A: DomainNet dataset samples for the class "Aircraft Carrier". B: Office-Home dataset samples for the class "Alarm Clock". C: Office-31 dataset samples for the class "Backpack".
• Figure 4: Overview of the Video Classification (A) and Object Detection (B and C) datasets used in our benchmarking comparisons. A: UCF101 with its $5$ major categories. B: Pool-And-Car sample images overlayed with pool and car detections. C: XView sample images overlayed with detections of "Small car", "Truck" and "Trailer" classes.
• Figure 5: Qualitative results for the Few-Shot Domain Adapt Image Classification task using the PACMAC prabhu2022adapting algorithm on the (A) DomainNet Sketch (B) Office-Home Art datasets.