Prompt-Based Bias Calibration for Better Zero/Few-Shot Learning of Language Models

TL;DR

This work proposes a null-input prompting method to calibrate intrinsic bias encoded in pre-trained LMs, leveraging a diverse set of auto-selected null-meaning inputs generated from GPT-4 to probe intrinsic bias of pre-trained LMs.

Abstract

Prompt-based learning is susceptible to intrinsic bias present in pre-trained language models (LMs), leading to sub-optimal performance in prompt-based zero/few-shot settings. In this work, we propose a null-input prompting method to calibrate intrinsic bias encoded in pre-trained LMs. Different from prior efforts that address intrinsic bias primarily for social fairness and often involve excessive computational cost, our objective is to explore enhancing LMs' performance in downstream zero/few-shot learning while emphasizing the efficiency of intrinsic bias calibration. Specifically, we leverage a diverse set of auto-selected null-meaning inputs generated from GPT-4 to probe intrinsic bias of pre-trained LMs. Utilizing the bias-reflected probability distribution, we formulate a distribution disparity loss for bias calibration, where we exclusively update bias parameters ($0.1\%$ of total parameters) of LMs towards equal probability distribution. Experimental results show that the calibration promotes an equitable starting point for LMs while preserving language modeling abilities. Across a wide range of datasets, including sentiment analysis and topic classification, our method significantly improves zero/few-shot learning performance of LMs for both in-context learning and prompt-based fine-tuning (on average $9\%$ and $2\%$, respectively).

Figures (8)

• Figure 1: We demonstrate our calibration method significantly improves classification performance of pre-trained LM. Upper: The pipeline of proposed null-input prompting method for intrinsic bias calibration targeting AGNews task zhang2015character. Lower left: Performance comparison of zero-shot in-context learning using: original LM (Orig. RoBERTa); calibrated (Calib.) LM with full model updates (WLM + BLM); calibrated LM with only BLM updates. Lower right: Case study illustrating that LM makes correct prediction after intrinsic bias calibration.
• Figure 2: Empirical experiments show the impact of calibration on zero-shot learning performance as the number of calibration batches increases (batch size is 32). The intersections of the curves and red vertical line signify the outcomes of the first calibration batch.
• Figure 3: t-SNE visualization for output representations of <mask> token. Left is obtained from original LM; Right is obtained from the LM after One-batch Calibration. Two colors denote the two classes in Subj task.
• Figure 4: Impact of calibration on downstream tasks shown through the changes with respect to baseline on each column. Each row shows the zero-shot performance of one task employing: original LM (first column; baseline), task-specific calibrated LM (diagonal), other-task calibrated LM (other places).
• Figure 5: Visualization of attention score by the depth of color in the connecting lines. We only show the attention between <mask> token and null-meaning input $x_{\text{null}}$. $Attn_{\textit{<mask>}}(x_{\text{null}})$ is the attention score of <mask> on $x_{\text{null}}$, averaged over encoder layers and attention heads. Left: Higher attention score indicates enhanced pattern extraction from $x_{\text{null}}$ which has higher $P_\textit{nsp}(x_{\text{null}}, \textit{ans})$.