The Unreasonable Effectiveness of Eccentric Automatic Prompts

TL;DR

Prompt formulation markedly affects LLM math performance, with 'positive thinking' prompts showing model- and data-dependent effects. The study demonstrates that automated prompt optimization via a DSPy-based approach typically outperforms hand-tuned prompts and generalizes better across evaluation sets. Automatic prompts can be highly unconventional yet yield superior results, underscoring the limits of manual prompt design. The work also highlights reproducibility challenges in LLM research and advocates for sharing prompts to enable replication and robust benchmarking.

Abstract

Large Language Models (LLMs) have demonstrated remarkable problem-solving and basic mathematics abilities. However, their efficacy is highly contingent on the formulation of the prompt. This study endeavors to quantify the influence of incorporating "positive thinking" into the system message of the prompt, then compare that to systematic prompt optimization. We assess the performance of 60 combinations of system message snippets, tested with and without Chain of Thought prompting, across three models with parameters ranging from 7 to 70 billion on the GSM8K dataset. Our findings reveal that results do not universally generalize across models. In most instances, the inclusion of "positive thinking" prompts positively affected model performance. Notably, however, Llama2-70B exhibited an exception when not utilizing Chain of Thought, as the optimal system message was found to be none at all. Given the combinatorial complexity, and thus computation time, of experimenting with hand-tuning prompts for large black-box models, we then compared the performance of the best "positive thinking" prompt against the output of systematic prompt optimization. We show that employing an automated prompt optimizer emerges as the most effective method for enhancing performance, even when working with smaller open-source models. Additionally, our findings reveal that the highest-scoring, automatically-optimized prompt exhibits a degree of peculiarity far beyond expectations.