Prompt-MII: Meta-Learning Instruction Induction for LLMs

TL;DR

Prompt-MII tackles the inefficiency of in-context learning by meta-learning a policy that converts a dataset's few-shot examples into a compact, reusable instruction. Using reinforcement learning and a model-specific meta-prompt template, it trains on thousands of diverse datasets and evaluates on unseen tasks, achieving performance comparable to 100-shot ICL while using far fewer tokens. It also outperforms explicit optimization baselines like APE and GEPA and demonstrates cross-model transfer with varying degrees of success. The approach offers a scalable, test-time-efficient path for adapting LLMs to new tasks through one-pass instruction induction and highlights the importance of meta-prompt design and model compatibility.

Abstract

A popular method to adapt large language models (LLMs) to new tasks is in-context learning (ICL), which is effective but incurs high inference costs as context length grows. In this paper we propose a method to perform instruction induction, where we take training examples and reduce them to a compact but descriptive prompt that can achieve performance comparable to ICL over the full training set. Specifically, we propose PROMPT-MII, a reinforcement learning (RL) based framework to meta-learn an instruction induction model that can generate compact instructions on the fly for an arbitrary new dataset. We train on over 3,000 diverse classification datasets from the HuggingFace hub, and evaluate on 90 unseen tasks. PROMPT-MII improves downstream model quality by 4-9 F1 points (10-20% relative), matching ICL performance while requiring 3-13x fewer tokens.

Figures (8)

• Figure 1: Classification task performance averaged over 90 datasets, using the Llama-3.1-8B-Instruct model. Prompt-MII achieves performance comparable to ICL while using $13\times$ fewer tokens.
• Figure 2: Overview of Prompt-MII. We train an Instruction Generator LLM's ability to perform instruction induction. At inference time, given dataset examples of an unseen task, it can automatically generate a reusable task instruction in a single pass, which then guides a black-box Instruction Follower LLM to make predictions.
• Figure 3: Performance vs prompt length comparison across different prompting methods. Prompt-MII (green diamonds) consistently outperforms other methods while using fewer tokens than ICL (blue triangles). Dashed lines connect ICL and trained methods for the same number of examples (n), demonstrating prompt compression while maintaining performance.
• Figure 4: Analysis of when Prompt-MII excels over ICL by per example token length.
• Figure 5: Cross-model transfer results showing large model instruction generation capabilities. Purple dashed lines connect larger model Prompt-MII-Zero performance (Llama3.1-405B and Qwen3-235B) to ICL baselines for the same number of examples, demonstrating that large models can generate effective instructions for smaller models to follow.