Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ATLAS : Adaptive Self-Evolutionary Research Agent with Task-Distributed Multi-LLM Supporters

Ujin Jeon, Jiyong Kwon, Madison Ann Sullivan, Caleb Eunho Lee, Guang Lin

TL;DR

ATLAS presents a robust framework for self-evolving AI agents by decomposing long-horizon learning into task-distributed supporter roles and an adaptive EvoDPO loop with phase-indexed references. The core algorithm integrates Direct Preference Optimization with a gated, KL-regularized reference update to combat drift and stagnation, formalized via a non-stationary contextual bandit regret analysis. Empirical results in non-stationary bandits and SciML PINN tasks demonstrate that ATLAS outperforms static baselines and simple ablations, improving stability and performance under changing environments. The work offers a principled path toward reliable, autonomous scientific problem solving and code generation in challenging regimes, while outlining safety and alignment considerations for autonomous self-improvement.

Abstract

Recent multi-LLM agent systems perform well in prompt optimization and automated problem-solving, but many either keep the solver frozen after fine-tuning or rely on a static preference-optimization loop, which becomes intractable for long-horizon tasks. We propose ATLAS (Adaptive Task-distributed Learning for Agentic Self-evolution), a task-distributed framework that iteratively develops a lightweight research agent while delegating complementary roles to specialized supporter agents for exploration, hyperparameter tuning, and reference policy management. Our core algorithm, Evolving Direct Preference Optimization (EvoDPO), adaptively updates the phase-indexed reference policy. We provide a theoretical regret analysis for a preference-based contextual bandit under concept drift. In addition, experiments were conducted on non-stationary linear contextual bandits and scientific machine learning (SciML) loss reweighting for the 1D Burgers' equation. Both results show that ATLAS improves stability and performance over a static single-agent baseline.

ATLAS : Adaptive Self-Evolutionary Research Agent with Task-Distributed Multi-LLM Supporters

TL;DR

ATLAS presents a robust framework for self-evolving AI agents by decomposing long-horizon learning into task-distributed supporter roles and an adaptive EvoDPO loop with phase-indexed references. The core algorithm integrates Direct Preference Optimization with a gated, KL-regularized reference update to combat drift and stagnation, formalized via a non-stationary contextual bandit regret analysis. Empirical results in non-stationary bandits and SciML PINN tasks demonstrate that ATLAS outperforms static baselines and simple ablations, improving stability and performance under changing environments. The work offers a principled path toward reliable, autonomous scientific problem solving and code generation in challenging regimes, while outlining safety and alignment considerations for autonomous self-improvement.

Abstract

Recent multi-LLM agent systems perform well in prompt optimization and automated problem-solving, but many either keep the solver frozen after fine-tuning or rely on a static preference-optimization loop, which becomes intractable for long-horizon tasks. We propose ATLAS (Adaptive Task-distributed Learning for Agentic Self-evolution), a task-distributed framework that iteratively develops a lightweight research agent while delegating complementary roles to specialized supporter agents for exploration, hyperparameter tuning, and reference policy management. Our core algorithm, Evolving Direct Preference Optimization (EvoDPO), adaptively updates the phase-indexed reference policy. We provide a theoretical regret analysis for a preference-based contextual bandit under concept drift. In addition, experiments were conducted on non-stationary linear contextual bandits and scientific machine learning (SciML) loss reweighting for the 1D Burgers' equation. Both results show that ATLAS improves stability and performance over a static single-agent baseline.
Paper Structure (66 sections, 5 theorems, 131 equations, 2 figures, 7 tables, 2 algorithms)

This paper contains 66 sections, 5 theorems, 131 equations, 2 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Proposed Methods
  3. ATLAS : Adaptive Task-distributed Learning for Agentic Self-evolution
  4. Self-evolving research agent.
  5. Task-Distributed Multi-LLM Supporters.
  6. Evolving DPO algorithm
  7. Direct Preference Optimization.
  8. Evolving DPO.
  9. Empirical KL estimate in the gate.
  10. Inspector score functional.
  11. Reference-improvement operator.
  12. Reference update with conservative acceptance.
  13. Theorem
  14. Theory--practice connection.
  15. Problem Formulation
  16. ...and 51 more sections

Key Result

Lemma 1.3

Under the margin condition with gap $\gamma > 0$, the number of switches of the oracle-optimal action satisfies

Figures (2)

  • Figure 1: ATLAS workflow. ATLAS (Adaptive Task-distributed Learning for Agentic Self-evolution) alternates between (i) exploration with a supporter agent to generate diverse candidates and a preference dataset, and (ii) EvoDPO updates consisting of DPO fine-tuning (strategist-guided) and reference promotion via an inspector gate based on score improvement and a KL budget.
  • Figure 2: Experimental Results across distinct domains. (a) Bandit Negative Mean Regret (NMR). (b) PINN Validation Loss (Log Scale). Shaded regions represent the Standard Error of the Mean (SEM) across 5 independent seeds.

Theorems & Definitions (10)

  • Lemma 1.3: Oracle Switching Budget
  • proof
  • Lemma 1.4: Local variation bound
  • proof
  • Lemma 1.5: Self-normalized inequality
  • proof
  • Lemma 1.6: Parameter estimation error bound
  • proof
  • Lemma 1.7: KL divergence bound
  • proof