System Prompt Optimization with Meta-Learning

TL;DR

This work introduces bilevel system prompt optimization and the MetaSPO framework, which jointly optimizes a universal, task-agnostic system prompt and per-task user prompts through an inner-outer loop structure. By meta-learning over a distribution of tasks, MetaSPO training yields a system prompt that generalizes to unseen tasks and robustly accommodates diverse user prompts, while iteratively refining task-specific user prompts to maintain synergy. Extensive experiments across 14 unseen tasks in 5 domains demonstrate strong unseen generalization and improved test-time adaptation, with MetaSPO achieving higher performance and greater efficiency than baselines such as SPRIG and default prompts. The results indicate that a well-optimized system prompt can serve as a strong, transferable foundation for LLM behavior, enabling rapid adaptation with limited data and compute in real-world settings.

Abstract

Large Language Models (LLMs) have shown remarkable capabilities, with optimizing their input prompts playing a pivotal role in maximizing their performance. However, while LLM prompts consist of both the task-agnostic system prompts and task-specific user prompts, existing work on prompt optimization has focused on user prompts specific to individual queries or tasks, and largely overlooked the system prompt that is, once optimized, applicable across different tasks and domains. Motivated by this, we introduce the novel problem of bilevel system prompt optimization, whose objective is to design system prompts that are robust to diverse user prompts and transferable to unseen tasks. To tackle this problem, we then propose a meta-learning framework, which meta-learns the system prompt by optimizing it over various user prompts across multiple datasets, while simultaneously updating the user prompts in an iterative manner to ensure synergy between them. We conduct experiments on 14 unseen datasets spanning 5 different domains, on which we show that our approach produces system prompts that generalize effectively to diverse user prompts. Also, our findings reveal that the optimized system prompt enables rapid adaptation even to unseen tasks, requiring fewer optimization steps for test-time user prompts while achieving improved performance.

Figures (18)

• Figure 1: Concept Figure. (A) The input prompt provided to LLMs typically consists of a task-agnostic system prompt, a task-specific user prompt, and a target example to handle. (B) Conventional Task-Specific Optimization focuses on optimizing user prompts for a single task but shows limited generalization to other tasks. (C) The goal of Bilevel System Prompt Optimization (Ours) is to enable the optimized system prompt to generalize effectively to unseen target tasks, for which we utilize a meta-learning framework to derive meta-knowledge from multiple source tasks.
• Figure 2: Overview of MetaSPO, which consists of the inner loop for user prompt optimization and the outer loop for system prompt optimization, operationalized through the meta-learning framework. (A) Inner Loop generates candidate user prompts by analyzing incorrectly predicted examples and then evaluates them with the system prompt to select refined prompts for individual tasks. (B) Outer Loop generates candidate system prompts by analyzing incorrect examples from all source tasks, and then evaluates them across various user prompts and tasks to ensure generalizability.
• Figure 2: Main Results on Test-Time Adaptation, where we optimize the user prompts with examples from target tasks, while fixing the system prompt. The average score for each domain is reported.
• Figure 3: Performance of user prompts with MetaSPO ($y$) and Default ($x$). Points over $y=x$ indicate the superiority of MetaSPO.
• Figure 4: Relative performance improvements of our MetaSPO over Default as a function of the source-target tasks similarity, where the similarity is measured by Bag-of-Words (BoW).