Do Prompts Reshape Representations? An Empirical Study of Prompting Effects on Embeddings
Cesar Gonzalez-Gutierrez, Dirk Hovy
TL;DR
This work investigates whether prompts improve internal LM representations by contextualizing input embeddings and evaluating task-specific information with probes. Using zero-shot templates across four classification tasks and three model types, the authors quantify how prompt embeddings differ from the original inputs and from random prompts. They find that prompting indeed alters embeddings, but improvements do not consistently track with prompt relevance, and random prompts can sometimes outperform relevant ones. The results suggest that embedding-level analyses capture only part of in-context learning dynamics and highlight the need for larger pre-training and possibly instruction-tuning to fully understand prompting effects on representations.
Abstract
Prompting is a common approach for leveraging LMs in zero-shot settings. However, the underlying mechanisms that enable LMs to perform diverse tasks without task-specific supervision remain poorly understood. Studying the relationship between prompting and the quality of internal representations can shed light on how pre-trained embeddings may support in-context task solving. In this empirical study, we conduct a series of probing experiments on prompt embeddings, analyzing various combinations of prompt templates for zero-shot classification. Our findings show that while prompting affects the quality of representations, these changes do not consistently correlate with the relevance of the prompts to the target task. This result challenges the assumption that more relevant prompts necessarily lead to better representations. We further analyze potential factors that may contribute to this unexpected behavior.