Evaluating the fairness of task-adaptive pretraining on unlabeled test data before few-shot text classification
Kush Dubey
TL;DR
This study interrogates whether pretraining on unlabeled test-set text inflates few-shot text classification performance. It introduces three estimators (acc_extra, acc_test, acc_base) and applies a hierarchical Bayesian analysis to 25 tasks across BERT, GPT-2, and Mistral 7B in zero-shot settings, revealing a robust pretraining boost when using independent unlabeled data while showing no consistent evaluation bias from test-set pretraining ($E[\text{acctest} - \text{accextra}] \approx 0$). The analysis demonstrates substantial within-task and cross-task variance, and emphasizes the necessity of repeated subsampling to obtain stable, model- and task-agnostic conclusions. The findings suggest that releasing unlabeled test-set text does not inherently bias benchmark evaluations, but underscore the importance of rigorous experimental design and transparency in few-shot NLP assessments, particularly for LLM-based evaluations.
Abstract
Few-shot learning benchmarks are critical for evaluating modern NLP techniques. It is possible, however, that benchmarks favor methods which easily make use of unlabeled text, because researchers can use unlabeled text from the test set to pretrain their models. Given the dearth of research on this potential problem, we run experiments to quantify the bias caused by pretraining on unlabeled test set text instead of on unlabeled, independently drawn text. Controlled few-shot and zero-shot experiments on 25 classification tasks and 3 language models -- BERT, GPT-2, and Mistral 7B -- do not find evidence of overoptimism. Furthermore, we demonstrate the importance of repeated subsampling when studying few-shot text classification, and recommend that few-shot learning benchmarks include multiple training folds. Code and data are available at https://github.com/kddubey/pretrain-on-test/.