Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution

Jiachen Jiang, Tianyu Ding, Zhihui Zhu

TL;DR

DeltaEvolve introduces a momentum-driven framework for LLM-guided program evolution by replacing full-code histories with semantic deltas. Framed as an EM process, it decouples the learning signal into context construction (M-step) and candidate sampling (E-step), with a multi-level memory and progressive disclosure. Across five scientific domains, DeltaEvolve achieves comparable or better solution quality while reducing token consumption by about 36–37% versus full-code baselines. The approach highlights semantic transferability of modification components and offers a practical path to scalable, token-efficient automated discovery.

Abstract

LLM-driven evolutionary systems have shown promise for automated science discovery, yet existing approaches such as AlphaEvolve rely on full-code histories that are context-inefficient and potentially provide weak evolutionary guidance. In this work, we first formalize the evolutionary agents as a general Expectation-Maximization framework, where the language model samples candidate programs (E-step) and the system updates the control context based on evaluation feedback (M-step). Under this view, constructing context via full-code snapshots constitutes a suboptimal M-step, as redundant implement details dilutes core algorithmic ideas, making it difficult to provide clear inspirations for evolution. To address this, we propose DeltaEvolve, a momentum-driven evolutionary framework that replaces full-code history with structured semantic delta capturing how and why modifications between successive nodes affect performance. As programs are often decomposable, semantic delta usually contains many effective components which are transferable and more informative to drive improvement. By organizing semantic delta through multi-level database and progressive disclosure mechanism, input tokens are further reduced. Empirical evaluations on tasks across diverse scientific domains show that our framework can discover better solution with less token consumption over full-code-based evolutionary agents.

DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution

TL;DR

DeltaEvolve introduces a momentum-driven framework for LLM-guided program evolution by replacing full-code histories with semantic deltas. Framed as an EM process, it decouples the learning signal into context construction (M-step) and candidate sampling (E-step), with a multi-level memory and progressive disclosure. Across five scientific domains, DeltaEvolve achieves comparable or better solution quality while reducing token consumption by about 36–37% versus full-code baselines. The approach highlights semantic transferability of modification components and offers a practical path to scalable, token-efficient automated discovery.

Abstract

LLM-driven evolutionary systems have shown promise for automated science discovery, yet existing approaches such as AlphaEvolve rely on full-code histories that are context-inefficient and potentially provide weak evolutionary guidance. In this work, we first formalize the evolutionary agents as a general Expectation-Maximization framework, where the language model samples candidate programs (E-step) and the system updates the control context based on evaluation feedback (M-step). Under this view, constructing context via full-code snapshots constitutes a suboptimal M-step, as redundant implement details dilutes core algorithmic ideas, making it difficult to provide clear inspirations for evolution. To address this, we propose DeltaEvolve, a momentum-driven evolutionary framework that replaces full-code history with structured semantic delta capturing how and why modifications between successive nodes affect performance. As programs are often decomposable, semantic delta usually contains many effective components which are transferable and more informative to drive improvement. By organizing semantic delta through multi-level database and progressive disclosure mechanism, input tokens are further reduced. Empirical evaluations on tasks across diverse scientific domains show that our framework can discover better solution with less token consumption over full-code-based evolutionary agents.
Paper Structure (59 sections, 9 equations, 14 figures, 4 tables, 1 algorithm)

This paper contains 59 sections, 9 equations, 14 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Evolutionary Framework
  4. Problem Definition
  5. The EM Interpretation
  6. Context Selection Dominates Scalar Feedback
  7. DeltaEvolve
  8. Motivation
  9. Multi-Level Database
  10. L1: Delta Summary ($\delta^{(1)}_t$).
  11. L2: Delta Plan Details ($\delta^{(2)}_t$).
  12. L3: Full Code ($p_t$).
  13. Progressive Disclosure Sampler
  14. Phase 1: Node Selection.
  15. Phase 2: Progressive Rendering.
  16. ...and 44 more sections

Figures (14)

  • Figure 1: Illustration of search dynamics under existing methods and $\mathop{\mathtt{DeltaEvolve}}\limits$. Existing methods store and reuse full programs, whereas $\mathop{\mathtt{DeltaEvolve}}\limits$ stores $\mathop{\mathtt{semantic\ delta}}\limits$ that capture what changed and why it worked, forming a momentum-like memory that provides more informative guidance for reuse.
  • Figure 2: Comparison between AlphaEvolve and $\mathop{\mathtt{DeltaEvolve}}\limits$ (ours) on black-box optimization over 100 iterations. The x-axis shows cumulative consumed tokens across all iterations, and y-axis shows the best achieved score. Point size indicates the number of top and diverse nodes included in the context. $\mathop{\mathtt{DeltaEvolve}}\limits$ consistently achieves higher scores with fewer tokens.
  • Figure 3: Comparison of the $\mathop{\mathtt{DeltaEvolve}}\limits$ pipeline with AlphaEvolve. $\mathop{\mathtt{DeltaEvolve}}\limits$ incorporates $\mathop{\mathtt{semantic\ delta}}\limits$ into the context window instead of full code.
  • Figure 4: Progressive Disclosure Sampler
  • Figure 5: System Prompt Template. Includes novel instructions requiring the model to output structured Delta Summaries (Level 1) and Delta Plans (Level 2) alongside code, enabling automated evolutionary logging.
  • ...and 9 more figures

Theorems & Definitions (1)

  • Definition 4.1: Compositional Programs