Self-Improvement of Large Language Models: A Technical Overview and Future Outlook

Abstract

As large language models (LLMs) continue to advance, improving them solely through human supervision is becoming increasingly costly and limited in scalability. As models approach human-level capabilities in certain domains, human feedback may no longer provide sufficiently informative signals for further improvement. At the same time, the growing ability of models to make autonomous decisions and execute complex actions naturally enables abstractions in which components of the model development process can be progressively automated. Together, these challenges and opportunities have driven increasing interest in self-improvement, where models autonomously generate data, evaluate outputs, and iteratively refine their own capabilities. In this paper, we present a system-level perspective on self-improving language models and introduce a unified framework that organizes existing techniques. We conceptualize the self-improvement system as a closed-loop lifecycle, consisting of four tightly coupled processes: data acquisition, data selection, model optimization, and inference refinement, along with an autonomous evaluation layer. Within this framework, the model itself plays a central role in driving each stage: collecting or generating data, selecting informative signals, updating its parameters, and refining outputs, while the autonomous evaluation layer continuously monitors progress and guides the improvement cycle across stages. Following this lifecycle perspective, we systematically review and analyze representative methods for each component from a technical standpoint. We further discuss current limitations and outline our vision for future research toward fully self-improving LLMs.

Figures (11)

• Figure 1: Vision of self-improved language models. Humans only bootstrap the system, after which the model autonomously performs many operations such as acquiring data, reflecting on its outputs, and iteratively refining its capabilities to improve itself, potentially enabling the system to evolve beyond human-level intelligence.
• Figure 2: Overview of the proposed self-improvement system. The system consists of four interconnected stages: (i) Data Acquisition, (ii) Data Selection, (iii) Model Optimization, and (iv) Inference Refinement. Through which the model autonomously improves its capabilities. An additional (v) Autonomous Evaluation module provides continuous feedback to monitor progress and guide long-term iterative improvement.
• Figure 3: A taxonomy of the self-improvement system of LLMs.
• Figure 4: Overview of data acquisition in the self-improvement system. This stage focuses on how a model autonomously acquires raw data or experiences that can be used for self-improvement. (i) Static Curation: The model collects and filters information from pre-existing datasets or knowledge sources to construct curated training data. (ii) Environment Interaction: The model actively interacts with external environments, issuing actions and receiving observations to form interaction trajectories. (iii): Synthetic Generation: The model generates new training data directly from its own parameters through prompting, transformation, or multi-model interaction.
• Figure 5: Overview of data selection in the self-improvement system. This stage focuses on how a model evaluates and selects high-quality data from raw data pools to support effective self-improvement. (i) Metric-Guided Scoring: The model applies predefined scoring metrics derived from model signals to rank and filter data. One-shot scoring computes scores once before training, while iterative re-scoring periodically refreshes scores as the model evolves. (ii) Adaptive Selection: A learnable selector dynamically chooses training data through an iterative loop of selection, evaluation, and selector update, continuously adapting the training distribution based on feedback.