Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Understanding In-Context Learning with Contrastive Demonstrations and Saliency Maps

Fuxiao Liu, Paiheng Xu, Zongxia Li, Yue Feng, Hyemi Song

TL;DR

This work probes how demonstration components influence in-context learning in large language models by constructing contrastive demonstrations (label flipping, input neutralization, complementary explanations) and analyzing their saliency maps with XNLP methods. It reveals scale-dependent effects: large LLMs can override flipped mappings, while input perturbations have more modest impact, and the benefits of explanations depend on the task. The findings offer practical guidance for curating demonstrations, especially for applications like ChatGPT, and emphasize that saliency patterns alone may not guarantee improved ICL performance across tasks. The study provides a public codebase to enable further exploration of demonstration design and saliency-based analyses.

Abstract

We investigate the role of various demonstration components in the in-context learning (ICL) performance of large language models (LLMs). Specifically, we explore the impacts of ground-truth labels, input distribution, and complementary explanations, particularly when these are altered or perturbed. We build on previous work, which offers mixed findings on how these elements influence ICL. To probe these questions, we employ explainable NLP (XNLP) methods and utilize saliency maps of contrastive demonstrations for both qualitative and quantitative analysis. Our findings reveal that flipping ground-truth labels significantly affects the saliency, though it's more noticeable in larger LLMs. Our analysis of the input distribution at a granular level reveals that changing sentiment-indicative terms in a sentiment analysis task to neutral ones does not have as substantial an impact as altering ground-truth labels. Finally, we find that the effectiveness of complementary explanations in boosting ICL performance is task-dependent, with limited benefits seen in sentiment analysis tasks compared to symbolic reasoning tasks. These insights are critical for understanding the functionality of LLMs and guiding the development of effective demonstrations, which is increasingly relevant in light of the growing use of LLMs in applications such as ChatGPT. Our research code is publicly available at https://github.com/paihengxu/XICL.

Towards Understanding In-Context Learning with Contrastive Demonstrations and Saliency Maps

TL;DR

This work probes how demonstration components influence in-context learning in large language models by constructing contrastive demonstrations (label flipping, input neutralization, complementary explanations) and analyzing their saliency maps with XNLP methods. It reveals scale-dependent effects: large LLMs can override flipped mappings, while input perturbations have more modest impact, and the benefits of explanations depend on the task. The findings offer practical guidance for curating demonstrations, especially for applications like ChatGPT, and emphasize that saliency patterns alone may not guarantee improved ICL performance across tasks. The study provides a public codebase to enable further exploration of demonstration design and saliency-based analyses.

Abstract

We investigate the role of various demonstration components in the in-context learning (ICL) performance of large language models (LLMs). Specifically, we explore the impacts of ground-truth labels, input distribution, and complementary explanations, particularly when these are altered or perturbed. We build on previous work, which offers mixed findings on how these elements influence ICL. To probe these questions, we employ explainable NLP (XNLP) methods and utilize saliency maps of contrastive demonstrations for both qualitative and quantitative analysis. Our findings reveal that flipping ground-truth labels significantly affects the saliency, though it's more noticeable in larger LLMs. Our analysis of the input distribution at a granular level reveals that changing sentiment-indicative terms in a sentiment analysis task to neutral ones does not have as substantial an impact as altering ground-truth labels. Finally, we find that the effectiveness of complementary explanations in boosting ICL performance is task-dependent, with limited benefits seen in sentiment analysis tasks compared to symbolic reasoning tasks. These insights are critical for understanding the functionality of LLMs and guiding the development of effective demonstrations, which is increasingly relevant in light of the growing use of LLMs in applications such as ChatGPT. Our research code is publicly available at https://github.com/paihengxu/XICL.
Paper Structure (34 sections, 5 figures)

This paper contains 34 sections, 5 figures.

Table of Contents

  1. abstract
  2. Introduction
  3. Background
  4. Understanding ICL
  5. Saliency Maps
  6. Gradient-based Methods
  7. Perturbation-based Methods
  8. Approach
  9. Experimental Set-up
  10. Dataset
  11. Demonstration Selection
  12. Label Flipping
  13. Input Neutralization
  14. Complementary Explanation
  15. Baseline LMs and Metric
  16. ...and 19 more sections

Figures (5)

  • Figure 1: An overview of three ways to build contrastive demonstrations - flipping labels, perturbing (neutralizing) input, and adding complementary explanations. The contrastive parts are colored in red.
  • Figure 2: Full prompts (demonstration + test example) used for original demonstration and three contrastive variants. Tokens are color-coded by saliency scores for GPT2 generated by IG. The red box in original and neutralized prompts indicates manually selected sentiment-indicative and sentiment-neutral terms that we used for saliency map comparison.
  • Figure 3: Model Performance under the four conditions, with four demonstrations given
  • Figure 4: Model Performance under the four conditions, with eight demonstrations given.
  • Figure 5: Full prompts (demonstration + test example) used for original demonstration and three contrastive variants. Tokens are color-coded by saliency scores for generated by LIME. The red box in original and neutralized prompts indicates manually selected sentiment-indicative and sentiment-neutral terms that we used for saliency map comparison.