EDGE: Efficient Data Selection for LLM Agents via Guideline Effectiveness

TL;DR

EDGE addresses data inefficiency in LLM-agent development by introducing Guideline Effectiveness ($GE$), a metric that quantifies the impact of guidelines on action generation in multi-turn tasks. The framework computes $GE$ from initial guidelines and an action-unfolding trajectory, then iteratively updates guidelines using low-$GE$ samples and generates high-quality SFT data with GPT-4 guided by the updated guidelines, all without requiring golden answers. Empirical results on WebShop and HotpotQA show substantial data reductions (e.g., up to 50% on WebShop and 25% on HotpotQA) while achieving competitive or superior performance, highlighting the importance of data quality over quantity. This work contributes a practical, human-in-the-loop data-selection pipeline for prompt engineering and open-source fine-tuning, offering a new lens on data quality for LLM-agent evaluation and deployment.

Abstract

Large Language Models (LLMs) have shown remarkable capabilities as AI agents. However, existing methods for enhancing LLM-agent abilities often lack a focus on data quality, leading to inefficiencies and suboptimal results in both fine-tuning and prompt engineering. To address this issue, we introduce EDGE, a novel approach for identifying informative samples without needing golden answers. We propose the Guideline Effectiveness (GE) metric, which selects challenging samples by measuring the impact of human-provided guidelines in multi-turn interaction tasks. A low GE score indicates that the human expertise required for a sample is missing from the guideline, making the sample more informative. By selecting samples with low GE scores, we can improve the efficiency and outcomes of both prompt engineering and fine-tuning processes for LLMs. Extensive experiments validate the performance of our method. Our method achieves competitive results on the HotpotQA and WebShop and datasets, requiring 75\% and 50\% less data, respectively, while outperforming existing methods. We also provide a fresh perspective on the data quality of LLM-agent fine-tuning.

Figures (5)

• Figure 1: The overall process of our method and an example of how guidelines correct LLM's behavior
• Figure 2: Main results. The best results are marked in bold and the second-best results are marked with underline. Results marked with † are reported in the original paper.
• Figure 3: Comparison of different data selection strategies on various training budgets.
• Figure 4: Examples of various actions' GE scores. The left side shows an example with a high GE score. The right side shows examples with lower GE scores on Webshop and HotpotQA. The red highlight indicates the reason for the lower GE value of the action.
• Figure 5: Example of a peice of our guideline on HotpotQA