Instances Need More Care: Rewriting Prompts for Instances with LLMs in the Loop Yields Better Zero-Shot Performance

TL;DR

PRoMPTed introduces an instance-level prompt rewriting framework with an LLM-in-the-loop, where a meta LLM iteratively rewrites a task prompt based on the current output of a task LLM. Across 13 datasets and 10 task types, PRoMPTed yields substantial zero-shot gains over naive prompting and the Output Refinement baseline, and its benefits generalize to GPT-3.5 and open-source LLMs, including cross-family supervision where a weaker model rewrites prompts for a stronger one. The approach leverages a meta-prompt corpus and few-shot demonstrations to guide rewriting, improving knowledge recall and safety/honesty in outputs. While highly effective, it encounters challenges in symbolic reasoning and certain harmful-prompt scenarios, pointing to avenues for future work in robustness, safety, and scaling to diverse modalities.

Abstract

Large language models (LLMs) have revolutionized zero-shot task performance, mitigating the need for task-specific annotations while enhancing task generalizability. Despite its advancements, current methods using trigger phrases such as "Let's think step by step" remain limited. This study introduces PRomPTed, an approach that optimizes the zero-shot prompts for individual task instances following an innovative manner of "LLMs in the loop". Our comprehensive evaluation across 13 datasets and 10 task types based on GPT-4 reveals that PRomPTed significantly outperforms both the naive zero-shot approaches and a strong baseline (i.e., "Output Refinement") which refines the task output instead of the input prompt. Our experimental results also confirmed the generalization of this advantage to the relatively weaker GPT-3.5. Even more intriguingly, we found that leveraging GPT-3.5 to rewrite prompts for the stronger GPT-4 not only matches but occasionally exceeds the efficacy of using GPT-4 as the prompt rewriter. Our research thus presents a huge value in not only enhancing zero-shot LLM performance but also potentially enabling supervising LLMs with their weaker counterparts, a capability attracting much interest recently. Finally, our additional experiments confirm the generalization of the advantages to open-source LLMs such as Mistral 7B and Mixtral 8x7B.

Figures (3)

• Figure 1: Overview of PRoMPTed, which iteratively prompts the zero-shot task LLM to produce an output and then leverages a separate meta LLM to rewrite the input prompt based on the current task output. The final answer is extracted from the latest task output when the meta LLM considers the current prompt to be sufficiently well-written.
• Figure 2: Performance of PRoMPTed using different LLMs as $\mathcal{M}_{meta}$ and $\mathcal{M}_{task}$. We observed consistent performance gain when applying PRoMPTed to GPT-3.5. More excitingly, using the weaker GPT-3.5 to rewrite prompts for the stronger GPT-4 ("PRoMPTed (Meta=GPT-3.5, Task=GPT-4)") yields on-par or even better performance than using GPT-4 for prompt rewriting.
• Figure 3: Performance of PRoMPTed with and without $\mathcal{M}_{task}$ in the loop.