Better by Comparison: Retrieval-Augmented Contrastive Reasoning for Automatic Prompt Optimization

TL;DR

This work tackles prompt optimization for LLMs by reframing it as retrieval-augmented contrastive reasoning. CRPO retrieves top-$k$ reference prompt–response exemplars from HelpSteer2 and uses two reasoning paradigms—Tiered Contrastive Reasoning and Multi-Metric Contrastive Reasoning—to synthesize optimized prompts without model updates. Empirical results on HelpSteer2 show CRPO consistently outperforms direct generation, CoT, and RAG baselines, with ablations confirming the value of explicit contrastive reasoning over prompt–response pairs. The approach advances practical prompt optimization by enhancing robustness, interpretability, and alignment across multiple evaluation dimensions.

Abstract

Automatic prompt optimization has recently emerged as a strategy for improving the quality of prompts used in Large Language Models (LLMs), with the goal of generating more accurate and useful responses. However, most prior work focuses on direct prompt refinement or model fine-tuning, overlooking the potential of leveraging LLMs' inherent reasoning capability to learn from contrasting examples. In this paper, we present Contrastive Reasoning Prompt Optimization (CRPO), a novel framework that formulates prompt optimization as a retrieval-augmented reasoning process. Our approach retrieves top k reference prompt-response pairs from the HelpSteer2 dataset, an open source collection where each response is annotated for helpfulness, correctness, coherence, complexity, and verbosity, and constructs two complementary optimization paradigms: (1) tiered contrastive reasoning, where the LLM compares high-, medium-, and low-quality exemplars (both prompts and responses) to refine its own generation through reflective reasoning, and (2) multi-metric contrastive reasoning, where the LLM analyzes the best exemplars along each evaluation dimension and integrates their strengths into an optimized prompt. By explicitly contrasting high and low quality exemplars, CRPO enables the model to deduce why certain prompts succeed while others fail, thereby achieving more robust and interpretable optimization. Experimental results on the HelpSteer2 benchmark demonstrate that CRPO significantly outperforms baselines. Our findings highlight the promise of contrastive, retrieval-augmented reasoning for advancing automatic prompt optimization.

Figures (3)

• Figure 1: Overview of CRPO: The framework of CRPO-Tiered Contrastive Reasoning(top) and CRPO-Multi-Metric Contrastive Reasoning(bottom).
• Figure 2: Ablation 1. Performance comparison of the simplified method vs CRPO variants. Overall stands for the average score of 5 metrics.
• Figure 3: Ablation 2. Overall score comparison across Top-K settings in the RAG stage of CRPO, averaged over five metrics.