EchoPrompt: Instructing the Model to Rephrase Queries for Improved In-context Learning

TL;DR

EchoPrompt introduces a simple, general prompting technique that prompts LLMs to rephrase the input query before answering, aiming to improve in-context learning across zero-shot and few-shot settings. The approach is implemented as a subtask and evaluated across multiple model families and task types, showing broad performance gains on numerical reasoning, reading comprehension, and logical reasoning. Ablation studies reveal that both the original and rephrased queries contribute to improvements, and increasing the number of rephrases can hurt due to repetition, indicating EchoPrompt functions as a query augmentation method rather than merely generating more text. Overall, EchoPrompt provides a practical, model-agnostic enhancement to prompting strategies, with robust results and a clear reproducibility plan, suggesting it as a valuable building block for future prompting pipelines.

Abstract

Language models are achieving impressive performance on various tasks by aggressively adopting inference-time prompting techniques, such as zero-shot and few-shot prompting. In this work, we introduce EchoPrompt, a simple yet effective approach that prompts the model to rephrase its queries before answering them. EchoPrompt is adapted for both zero-shot and few-shot in-context learning with standard and chain-of-thought prompting. Experimental results show that EchoPrompt yields substantial improvements across all these settings for four families of causal language models. These improvements are observed across various numerical reasoning (e.g. GSM8K, SVAMP), reading comprehension (e.g. DROP), and logical reasoning (e.g. Coin Flipping) tasks. On average, EchoPrompt improves the Zero-shot-CoT performance of code-davinci-002 by 5% in numerical tasks and 13% in reading comprehension tasks. We investigate the factors contributing to EchoPrompt's effectiveness through ablation studies, which reveal that both the original query and the model-generated rephrased version are instrumental in its performance gains. Our empirical results indicate that EchoPrompt is an effective technique that enhances in-context learning performance. We recommend incorporating EchoPrompt into various baseline prompting strategies to achieve performance boosts.

Figures (4)

• Figure 1: Comparison of prompts in Zero-shot-CoT with and without EchoPrompt, highlighting the modification in prompts. Zero-shot-CoT with EchoPrompt uses the prompt "Let's repeat the question and also think step by step" to aid the model in recalling the query before solving it.
• Figure 2: Example of rephrases used for the proposed rephrase structures in EchoPrompt in few-shot prompting exemplars. The Rephrases of exemplars are generated using ChatGPT based on prompts in Table-\ref{['table:rephrase_instructions']}.
• Figure 3: Example of EchoPrompt with Compound Sentences. Standard Prompting approach showcases exemplars with queries and corresponding answering formats. In contrast, the EchoPrompt incorporates a Query-Rephrase step, where the exemplars showcase a rephrased version of the query along with the answering format.
• Figure 4: Performance summary of EchoPrompt with repetition in zero-shot and compound sentence rephrasing in few-shot settings. Darker colored bars show EchoPrompt augmented with the baseline method. EchoPrompt consistently achieves performance gains across different prompting strategies, particularly in zero-shot scenarios. For details, see Table-\ref{['table:summary']} in Appendix.