Generalizable Prompt Learning of CLIP: A Brief Overview
Fangming Cui, Yonggang Zhang, Xuan Wang, Xule Wang, Liang Xiao
TL;DR
This work surveys generalizable prompt learning for CLIP in few-shot settings across 15 datasets, focusing on how learnable prompts—in text and image modalities—can improve cross-domain and novel-class generalization over traditional prompt templates. It synthesizes a broad range of methods (e.g., CoOp, CoCoOp, MaPLe, QNet, DePT, AAPL, and multimodal approaches) and experimental regimes (base-to-novel, cross-dataset, and domain generalization) using a ViT-B/16 CLIP backbone. The analysis highlights bottlenecks in base-class performance, modest gains in cross-domain tasks, and gaps in very low-shot scenarios, pointing to robustness and generalization as key future challenges. Overall, the paper provides a practical reference for newcomers and researchers aiming to transfer and extend prompt-learning techniques to diverse vision-language tasks and datasets.
Abstract
Existing vision-language models (VLMs) such as CLIP have showcased an impressive capability to generalize well across various downstream tasks. These models leverage the synergy between visual and textual information, enabling them to understand and reason about the content present in images and text in a unified manner. This article provides a brief overview of CLIP based on few-shot prompt learning, including experimental data and technical characteristics of some methods. The purpose of this review is to provide a reference for researchers who have just started their research in generalizable prompting of CLIP through few-shot training for classification across 15 datasets and also to facilitate the integration of this field by researchers in other downstream tasks.