Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows

TL;DR

The paper argues that optimization problem solving can migrate from human-centric pipelines to evolutionary agentic workflows powered by foundation models and evolutionary search. It formalizes the optimization space as $\mathcal{O} := \mathcal{P} \otimes \mathcal{F} \otimes \mathcal{A} \otimes \mathcal{H}$ and presents a foundation-agent architecture (Memory, Reasoning, World Modeling with LLMs, and Action) complemented by an evolutionary framework that manages distributed populations and human-centered evaluation. Through case studies on agentic VM scheduling and ADMM hyperparameter tuning, the approach demonstrates autonomous problem formulation, algorithm design, and parameter adaptation that adapt to real-world industrial conditions. Limitations include theoretical verification, inference costs, and scalability, with proposed remedies like automated proofs, model compression, and HPC-enabled evaluation. Overall, the work outlines a viable path to scalable, adaptive optimization that integrates domain knowledge with principled search to accelerate industrial deployment of advanced optimization methods.

Abstract

This position paper argues that optimization problem solving can transition from expert-dependent to evolutionary agentic workflows. Traditional optimization practices rely on human specialists for problem formulation, algorithm selection, and hyperparameter tuning, creating bottlenecks that impede industrial adoption of cutting-edge methods. We contend that an evolutionary agentic workflow, powered by foundation models and evolutionary search, can autonomously navigate the optimization space, comprising problem, formulation, algorithm, and hyperparameter spaces. Through case studies in cloud resource scheduling and ADMM parameter adaptation, we demonstrate how this approach can bridge the gap between academic innovation and industrial implementation. Our position challenges the status quo of human-centric optimization workflows and advocates for a more scalable, adaptive approach to solving real-world optimization problems.

Figures (5)

• Figure 1: Comparative frameworks in optimization problem solving: human expertise and evolutionary agentic workflow.
• Figure 2: Automate optimization formulation, algorithmic design and hyperparameter selection for the transportation problem with evolutionary agentic workflow.
• Figure 3: Evolutionary agentic workflow for VM scheduling.
• Figure 4: The perforcement of the discovered algorithm during traing on scenario with VM size $50$.
• Figure 5: Evolutionary agentic workflow for ADMM.