Divide, Optimize, Merge: Fine-Grained LLM Agent Optimization at Scale

TL;DR

Fine-Grained LLM Agent Optimization (FGO) tackles scalability bottlenecks in LLM-based agent optimization by partitioning large training data into subsets, optimizing each subset independently, and progressively merging results to form a global model. The framework preserves task-specific patterns while enabling parallel processing and mitigating context-window constraints, yielding consistent gains (8.3%–38.1%) and substantial token savings across ALFWorld, LogisticsQA, and GAIA. Ablation studies show the crucial role of progressive merging and demonstrate robustness to partitioning strategies and backbone models, with manageable increases in token usage due to validation across merged tasks. Overall, FGO provides a practical, scalable approach for optimizing increasingly complex agentic systems with large-scale data.

Abstract

LLM-based optimization has shown remarkable potential in enhancing agentic systems. However, the conventional approach of prompting LLM optimizer with the whole training trajectories on training dataset in a single pass becomes untenable as datasets grow, leading to context window overflow and degraded pattern recognition. To address these challenges, we propose Fine-Grained Optimization (FGO), a scalable framework that divides large optimization tasks into manageable subsets, performs targeted optimizations, and systematically combines optimized components through progressive merging. Evaluation across ALFWorld, LogisticsQA, and GAIA benchmarks demonstrate that FGO outperforms existing approaches by 1.6-8.6% while reducing average prompt token consumption by 56.3%. Our framework provides a practical solution for scaling up LLM-based optimization of increasingly sophisticated agent systems. Further analysis demonstrates that FGO achieves the most consistent performance gain in all training dataset sizes, showcasing its scalability and efficiency.

Figures (8)

• Figure 1: Agent optimization approaches. (a) Basic agent execution process. (b) Traditional all-at-once optimization faces context overflow and inferior performance with large trajectory data. (c) Our method: divide-and-conquer optimization with progressive merging enables scalable processing of large datasets.
• Figure 2: Illustration of FGO's optimization pipeline. The system operates in three stages: (1) Divide: the full dataset is split into manageable subsets, (2) Optimize: parallel optimization is performed on each subset using LLM-based optimization with multiple iterations, and (3) Merge: optimized modules are progressively combined using recursive clustering to produce the final optimal agent system.
• Figure 3: Analysis of the number of training tasks. We run optimization on varied training dataset sizes and plot the performance. FGO achieves best performance in all training settings, and demonstrate a steady increase as the training taskset size increases.
• Figure 4: Comparison of prompt token efficiency across different optimization methods on ALFWorld, LogisticsQA, and GAIA. Each panel plots the trained agent's performance against the total prompt tokens consumed during optimization. Circle diameters are proportional to the optimization token consumption, with crosses (+) indicating circle centers.
• Figure 5: Ablation study on the number of divided subsets. Most parameter settings achieve similar performance, with varying time for optimization.