Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Small Models are Valuable Plug-ins for Large Language Models

Canwen Xu, Yichong Xu, Shuohang Wang, Yang Liu, Chenguang Zhu, Julian McAuley

TL;DR

This work introduces Super In-Context Learning (SuperICL), a framework that pairs a black-box LLM with locally fine-tuned plug-in models to improve supervised task performance while mitigating ICL instability. By fine-tuning a small plug-in, constructing confidence-informed contexts, and letting the LLM override plug-in predictions when appropriate, SuperICL achieves superior results on GLUE and competitive cross-lingual gains on XNLI. The approach enhances interpretability through explanations and shows robustness improvements, albeit with sensitivity to plug-in quality and adversarial vulnerabilities. The study demonstrates a practical paradigm for leveraging both cloud-based LLMs and local models to advance supervised learning under context limits and resource constraints.

Abstract

Large language models (LLMs) such as GPT-3 and GPT-4 are powerful but their weights are often publicly unavailable and their immense sizes make the models difficult to be tuned with common hardware. As a result, effectively tuning these models with large-scale supervised data can be challenging. As an alternative, In-Context Learning (ICL) can only use a small number of supervised examples due to context length limits. In this paper, we propose Super In-Context Learning (SuperICL) which allows black-box LLMs to work with locally fine-tuned smaller models, resulting in superior performance on supervised tasks. Our experiments demonstrate that SuperICL can improve performance beyond state-of-the-art fine-tuned models while addressing the instability problem of in-context learning. Furthermore, SuperICL can enhance the capabilities of smaller models, such as multilinguality and interpretability.

Small Models are Valuable Plug-ins for Large Language Models

TL;DR

This work introduces Super In-Context Learning (SuperICL), a framework that pairs a black-box LLM with locally fine-tuned plug-in models to improve supervised task performance while mitigating ICL instability. By fine-tuning a small plug-in, constructing confidence-informed contexts, and letting the LLM override plug-in predictions when appropriate, SuperICL achieves superior results on GLUE and competitive cross-lingual gains on XNLI. The approach enhances interpretability through explanations and shows robustness improvements, albeit with sensitivity to plug-in quality and adversarial vulnerabilities. The study demonstrates a practical paradigm for leveraging both cloud-based LLMs and local models to advance supervised learning under context limits and resource constraints.

Abstract

Large language models (LLMs) such as GPT-3 and GPT-4 are powerful but their weights are often publicly unavailable and their immense sizes make the models difficult to be tuned with common hardware. As a result, effectively tuning these models with large-scale supervised data can be challenging. As an alternative, In-Context Learning (ICL) can only use a small number of supervised examples due to context length limits. In this paper, we propose Super In-Context Learning (SuperICL) which allows black-box LLMs to work with locally fine-tuned smaller models, resulting in superior performance on supervised tasks. Our experiments demonstrate that SuperICL can improve performance beyond state-of-the-art fine-tuned models while addressing the instability problem of in-context learning. Furthermore, SuperICL can enhance the capabilities of smaller models, such as multilinguality and interpretability.
Paper Structure (26 sections, 3 figures, 9 tables)

This paper contains 26 sections, 3 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. In-Context Learning
  4. Language Model Plug-ins
  5. Super In-Context Learning
  6. Plug-in Model Fine-tuning
  7. Context Construction
  8. Inference
  9. Experiments
  10. Experimental Settings
  11. Benchmarks
  12. Large Language Model and Plug-ins
  13. Main Results
  14. GLUE
  15. XNLI
  16. ...and 11 more sections

Figures (3)

  • Figure 1: The workflow of ICL and SuperICL. There are three steps in SuperICL: (1) A context is constructed by randomly sampling from the training data and incorporating the plug-in model's predictions, including predicted labels and their corresponding confidence scores. (2) The test input is concatenated after the context, with the plug-in model's prediction attached. (3) Finally, a language model generates the final prediction along with an optional explanation.
  • Figure 2: Effect of plug-in model confidence for overrides. The figure is the distribution of RoBERTa confidence for all examples (blue) and examples with a final prediction overridden by GPT-3.5 (orange) on MRPC.
  • Figure 3: Effect of number of examples on the performance of ICL and SuperICL. The results are averages of three runs.