Active Learning Helps Pretrained Models Learn the Intended Task

TL;DR

<3-5 sentence high-level summary> Task ambiguity occurs when training data fails to specify the intended behavior for all inputs. The authors test whether pretrained models can resolve ambiguity through active learning, querying informative examples. They show that uncertainty-based AL with pretrained models achieves substantial label-efficiency gains across vision and text tasks, while non-pretrained baselines often gain little or degrade. They attribute these gains to a pretrained feature space in which disambiguating attributes are more linearly separable, though failures arise in domains distant from pretraining data. This suggests pretraining shapes AL behavior and can improve reliability under distribution shift and label scarcity.

Abstract

Models can fail in unpredictable ways during deployment due to task ambiguity, when multiple behaviors are consistent with the provided training data. An example is an object classifier trained on red squares and blue circles: when encountering blue squares, the intended behavior is undefined. We investigate whether pretrained models are better active learners, capable of disambiguating between the possible tasks a user may be trying to specify. Intriguingly, we find that better active learning is an emergent property of the pretraining process: pretrained models require up to 5 times fewer labels when using uncertainty-based active learning, while non-pretrained models see no or even negative benefit. We find these gains come from an ability to select examples with attributes that disambiguate the intended behavior, such as rare product categories or atypical backgrounds. These attributes are far more linearly separable in pretrained model's representation spaces vs non-pretrained models, suggesting a possible mechanism for this behavior.

Figures (14)

• Figure 1: Active learning can resolve task ambiguity in datasets. Here, the provided training data leaves the model unsure of the intended task: is it to predict the shape or the color of the object? Pretraining enables models to identify and weigh various rich features, eliciting labels from informative examples (e.g. blue squares) that clarify the user's intention.
• Figure 2: Uncertainty sampling outperforms random sampling on all datasets, especially on minority classes. Class-balanced accuracies displayed for \ref{['fig:line-iwc-minority']}. Shaded regions represent 95% CIs (Gaussian approx.).
• Figure 3: All types of uncertainty sampling outperform random sampling on iWildCam. Class 0 represents the majority class in iWildCam (no animal present).
• Figure 4: Uncertainty sampling identifies and upsamples disambiguating examples. For both Waterbirds and Treeperson, uncertainty sampling selectively acquires examples where the spurious and core features disagree. Y-axis: frequency of class in acquisitions. Oversampling is visible for subgroups where uncertainty sampling acquires examples above random chance.
• Figure 5: Uncertainty sampling upsamples both visible and latent minority subgroups. Fraction of Amazon examples acquired by random and uncertainty sampling, stratified by star rating and product category. Upsampling is visible when the bar for uncertainty sampling is greater than the base prevalance in the unlabeled dataset available during training. Uncertainty sampling preferentially acquires examples with lower star ratings and rarer product categories, despite the latter attribute not being visible to the model. Note the separate y-axis for product categories 1 and 2 in (b).