GLaPE: Gold Label-agnostic Prompt Evaluation and Optimization for Large Language Model

TL;DR

GLaPE tackles the problem that prompt performance in large language models is highly sensitive and that relying on gold labels restricts practical prompt optimization. It presents a gold label-agnostic evaluation framework that combines self-consistency with mutual-consistency refinement, formalized via $L_{self}$, $L_{refine}$, and $L_{total}$ with $L_{total}=\alpha L_{self}+(1-\alpha)L_{refine}$ and $\alpha=0.5$, initializing $f_i=c_i$ and using gradient descent with learning rate $0.05$ to obtain robust scores without gold labels. These scores guide a GLaPE-based prompt optimization that matches the performance of accuracy-based methods like OPRO across eight reasoning datasets and generalizes across several LLMs (e.g., Mistral-7B, Llama3-8B, Gemma-2-9B). The approach demonstrates that gold label-agnostic evaluation can effectively drive gradient-free prompt optimization in private LLM scenarios, with publicly available code, while also revealing limitations on datasets where consistent errors dominate and suggesting avenues for improvement. Overall, GLaPE offers a practical, scalable alternative to gold-label reliance for prompt design in diverse reasoning tasks.

Abstract

Despite the rapid progress of large language models (LLMs), their task performance remains sensitive to prompt design. Recent studies have explored leveraging the LLM itself as an optimizer to identify optimal prompts that maximize task accuracy. However, when evaluating prompts, such approaches heavily rely on elusive manually annotated gold labels to calculate task accuracy for each candidate prompt, which hinders the widespread implementation and generality. To overcome the limitation, this work proposes a gold label-agnostic prompt evaluation (GLaPE) to alleviate dependence on gold labels. Motivated by the observed correlation between self-consistency and the accuracy of the answer, we adopt self-consistency as the initial evaluation score. Subsequently, we refine the scores of prompts producing identical answers to be mutually consistent. Experimental results show that GLaPE provides reliable evaluations uniform with accuracy, even in the absence of gold labels. Moreover, on six popular reasoning tasks, our GLaPE-based prompt optimization yields effective prompts comparable to accuracy-based ones. The code is publicly available at https://github.com/thunderous77/GLaPE.

Figures (5)

• Figure 1: Sketch of prompt optimization utilizing the LLM as an optimizer LLMasOPT, featuring distinct prompt evaluation metrics based on: (a) accuracy or (b) our proposed GLaPE. The texts are favorably read in colors. Blue: gold label, Yellow: most frequent answer, Green: high score, Red: low score, Purple: prompt evaluation.
• Figure 2: The schematic representation of our GLaPE method integrating self-consistency (SC) evaluation and mutual-consistency (MC) refinement. This sketch illustrates how our method assesses the prompts in Figure \ref{['fig:sketch of LLMasOPT']}; computation details are provided in Appendix \ref{['sec:computation of GLaPE']}. Notably, we observed that prompt3, as indicated by the red marker, produces an incorrect answer with high self-consistency (70%). Through the mutual-consistency refinement, our GLaPE score experiences a decrease of $20.0$, rendering it more discernible when compared to prompt1 and prompt2. The texts are favorably read in colors of background. Blue: self-consistency, Purple: mutual-consistency refinement, Green: answer "31" (gold label), Orange: answer "36", Yellow: GLaPE metric.
• Figure 3: SC-Accuracy Graph for Prompts. Each prompt is represented as a point on the graph, where the x-coordinate signifies self-consistency and the y-coordinate signifies accuracy.
• Figure 4: SC-Accuracy Graph v.s. GLaPE-Accuracy Graph for Prompts on Multiarith.
• Figure 5: Some examples in StrategyQA dataset where the LLM consistently provides inaccurate responses.