EnviSAgE: A Survey of Environment Scaling for Qualitative Agentic Experience Collection

TL;DR

<3-5 sentence high-level summary> EnviSAgE argues that scaling environmental context is essential for training LLM-based agents to exhibit adaptive, long-horizon behavior, treating environments as active producers of experiential data through a Generation-Execution-Feedback loop. It offers a three-stage taxonomy—Task Generation, Task Execution, and Feedback—with concrete scaling dimensions (complexity, dynamics, diversity; interactivity, realism; density, granularity, automation, objectivity, robustness) and surveys representative methods, frameworks, and benchmarks. The paper analyzes implementation challenges (notably generator-verifier asymmetry), cross-domain applications, and open research questions, providing a structured roadmap for advancing agent intelligence via richer, more realistic environments. It concludes with forward-looking directions including co-evolution with external tools, generator-verifier synergy, and open-ended multi-agent environments to accelerate scalable, safe, and generalizable agent capabilities.

Abstract

LLM-based agents can autonomously accomplish complex tasks across various domains. However, to further cultivate capabilities such as adaptive behavior and long-term decision-making, training on static datasets built from human-level knowledge is insufficient. These datasets are costly to construct and lack both dynamism and realism. A growing consensus is that agents should instead interact directly with environments and learn from experience through reinforcement learning. We formalize this iterative process as the Generation-Execution-Feedback (GEF) loop, where environments generate tasks to challenge agents, return observations in response to agents' actions during task execution, and provide evaluative feedback on rollouts for subsequent learning. Under this paradigm, environments function as indispensable producers of experiential data, highlighting the need to scale them toward greater complexity, realism, and interactivity. In this survey, we systematically review representative methods for environment scaling from a pioneering environment-centric perspective and organize them along the stages of the GEF loop, namely task generation, task execution, and feedback. We further analyze implementation frameworks, challenges, and applications, consolidating fragmented advances and outlining future research directions for agent intelligence.

Figures (5)

• Figure 1: Experience arises from the Generation-Execution-Feedback (GEF) loop, where environments generate tasks, agents execute them, and environments evaluate and filter useful experience for RL training.
• Figure 2: GEF-aligned taxonomy of environment scaling with dimensions for Task Generation, Task Execution, and Feedback. Representative works are illustrated as leaves on the branches.
• Figure 3: Illustration of environment scaling in the task generation and task execution stages, using the example of conference scheduling. Given a user intent, the environment produces a set of tasks for the agent to complete. Scaling in the task generation stage covers complexity scaling, dynamic scaling, and diversity scaling, while in the task execution stage scaling encompasses interactivity scaling and realism scaling.
• Figure 4: Illustration of environment scaling in the feedback stage using a conference-scheduling example. The agent first executes tasks in the environment and produces action-observation trajectories. The environment then evaluates these trajectories and returns feedback, yielding the experience used to train the agent. Scaling in the feedback stage covers density, granularity, automation, objectivity, and robustness.
• Figure 5: Generator-verifier asymmetry challenge.