GALOT: Generative Active Learning via Optimizable Zero-shot Text-to-image Generation

TL;DR

The paper addresses data efficiency in active learning by coupling zero-shot text-to-image diffusion with guided sample selection. It formalizes text embedding selection via a constrained objective that optimizes acquisition over generated samples, enabling the generation of informative data and pseudo-labels for end-to-end text-to-model training. GALOT demonstrates strong empirical gains across CIFAR-10/100 and TinyImageNet, with data reuse across multiple architectures, and substantially reduces labeling costs by leveraging synthetic data. This approach broadens the practical impact of active learning by enabling a text-to-model pipeline with versatile, annotation-free data generation.

Abstract

Active Learning (AL) represents a crucial methodology within machine learning, emphasizing the identification and utilization of the most informative samples for efficient model training. However, a significant challenge of AL is its dependence on the limited labeled data samples and data distribution, resulting in limited performance. To address this limitation, this paper integrates the zero-shot text-to-image (T2I) synthesis and active learning by designing a novel framework that can efficiently train a machine learning (ML) model sorely using the text description. Specifically, we leverage the AL criteria to optimize the text inputs for generating more informative and diverse data samples, annotated by the pseudo-label crafted from text, then served as a synthetic dataset for active learning. This approach reduces the cost of data collection and annotation while increasing the efficiency of model training by providing informative training samples, enabling a novel end-to-end ML task from text description to vision models. Through comprehensive evaluations, our framework demonstrates consistent and significant improvements over traditional AL methods.

Figures (10)

• Figure 1: Overview of GALOT. The task-related text is first converted into the text embedding. Then, it iteratively executes 1) optimizing the text embedding according to the AL criteria based on the current model's output, 2) generating data samples with optimized text embedding, and 3) training the model with generated data and pseudo label. GALOT train the vision model from text inputs.
• Figure 2: GALOT Workflow for Each Active Learning Cycle. In each active learning cycle, the data sample is generated using pre-trained T2I models with the embedding $s$ and pseudo label. Optionally combined with the traditional AL with real datasets as a complement, the generated data can be used to train a model. The embedding is then optimized according to the updated model via the gradients of the AL acquisition.
• Figure 3: Comparison of Different Baselines.
• Figure 4: Text-to-image Generation Accuracy (Human Evaluated) vs. $\epsilon$ with Different Templates.
• Figure 5: Accuracy vs Acquisition Methods $\sigma_{GAL}$.

Theorems & Definitions (2)

• Proposition 1
• proof