Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement

Xunjian Yin, Xinyi Wang, Liangming Pan, Li Lin, Xiaojun Wan, William Yang Wang

TL;DR

The paper tackles rigidity in hand-designed and fixed meta-learning agents by introducing Gödel Agent, a self-referential framework that uses LLM-driven self-modification to iteratively rewrite its own code. It formalizes a recursive update system for both policy and learning algorithm, and implements runtime memory modification with prompting to enable continuous self-improvement. Across multiple reasoning and calculation benchmarks, Gödel Agent achieves competitive or superior performance with faster convergence and demonstrated robustness, particularly in open-ended scenarios. The work lays groundwork for autonomous, self-improving AI agents while discussing limitations, safety, and directions for future development.

Abstract

The rapid advancement of large language models (LLMs) has significantly enhanced the capabilities of AI-driven agents across various tasks. However, existing agentic systems, whether based on fixed pipeline algorithms or pre-defined meta-learning frameworks, cannot search the whole agent design space due to the restriction of human-designed components, and thus might miss the globally optimal agent design. In this paper, we introduce Gödel Agent, a self-evolving framework inspired by the Gödel machine, enabling agents to recursively improve themselves without relying on predefined routines or fixed optimization algorithms. Gödel Agent leverages LLMs to dynamically modify its own logic and behavior, guided solely by high-level objectives through prompting. Experimental results on mathematical reasoning and complex agent tasks demonstrate that implementation of Gödel Agent can achieve continuous self-improvement, surpassing manually crafted agents in performance, efficiency, and generalizability.

Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement

TL;DR

The paper tackles rigidity in hand-designed and fixed meta-learning agents by introducing Gödel Agent, a self-referential framework that uses LLM-driven self-modification to iteratively rewrite its own code. It formalizes a recursive update system for both policy and learning algorithm, and implements runtime memory modification with prompting to enable continuous self-improvement. Across multiple reasoning and calculation benchmarks, Gödel Agent achieves competitive or superior performance with faster convergence and demonstrated robustness, particularly in open-ended scenarios. The work lays groundwork for autonomous, self-improving AI agents while discussing limitations, safety, and directions for future development.

Abstract

The rapid advancement of large language models (LLMs) has significantly enhanced the capabilities of AI-driven agents across various tasks. However, existing agentic systems, whether based on fixed pipeline algorithms or pre-defined meta-learning frameworks, cannot search the whole agent design space due to the restriction of human-designed components, and thus might miss the globally optimal agent design. In this paper, we introduce Gödel Agent, a self-evolving framework inspired by the Gödel machine, enabling agents to recursively improve themselves without relying on predefined routines or fixed optimization algorithms. Gödel Agent leverages LLMs to dynamically modify its own logic and behavior, guided solely by high-level objectives through prompting. Experimental results on mathematical reasoning and complex agent tasks demonstrate that implementation of Gödel Agent can achieve continuous self-improvement, surpassing manually crafted agents in performance, efficiency, and generalizability.
Paper Structure (26 sections, 2 equations, 4 figures, 3 tables)

This paper contains 26 sections, 2 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Self-Referential Gödel Agent
  4. Gödel Agent Implementation
  5. Implementation Details
  6. Additional Designs
  7. Experiments
  8. Baseline Methods
  9. Experimental Settings
  10. Experimental Results and Analysis
  11. Analysis
  12. Analysis of Initial Tools
  13. Robustness Analysis of the Agent
  14. Case Study: Game of 24
  15. Discussions and Future Directions
  16. ...and 11 more sections

Figures (4)

  • Figure 1: An illustration of our implementation of Gödel Agent. It employs monkey patching to directly read and modify its own code in runtime memory, enabling self-awareness and self-modification.
  • Figure 2: The number of actions taken by Gödel Agent varies across different tasks.
  • Figure 3: (a) One representative example of Game of 24. (b) Accuracy progression for different initial policies.
  • Figure 4: Accuracy progression for Gödel Agent and random sampling.