Active Zero: Self-Evolving Vision-Language Models through Active Environment Exploration

TL;DR

Active-Zero tackles data-efficiency issues in self-play for vision-language models by enabling active exploration of open-world visual environments. It introduces a tri-agent loop (Searcher, Questioner, Solver) optimized with Group Relative Policy Optimization to form a closed auto-curriculum that co-evolves data acquisition and reasoning. Across 12 benchmarks and two model scales, Active-Zero yields consistent gains in both reasoning-intensive tasks and general visual understanding, outperforming existing self-play baselines. The work demonstrates that shifting from static datasets to active, uncertainty-driven data collection is a scalable path toward adaptive, self-evolving multimodal reasoning systems.

Abstract

Self-play has enabled large language models to autonomously improve through self-generated challenges. However, existing self-play methods for vision-language models rely on passive interaction with static image collections, resulting in strong dependence on initial datasets and inefficient learning. Without the ability to actively seek visual data tailored to their evolving capabilities, agents waste computational effort on samples that are either trivial or beyond their current skill level. To address these limitations, we propose Active-Zero, a framework that shifts from passive interaction to active exploration of visual environments. Active-Zero employs three co-evolving agents: a Searcher that retrieves images from open-world repositories based on the model's capability frontier, a Questioner that synthesizes calibrated reasoning tasks, and a Solver refined through accuracy rewards. This closed loop enables self-scaffolding auto-curricula where the model autonomously constructs its learning trajectory. On Qwen2.5-VL-7B-Instruct across 12 benchmarks, Active-Zero achieves 53.97 average accuracy on reasoning tasks (5.7% improvement) and 59.77 on general understanding (3.9% improvement), consistently outperforming existing self-play baselines. These results highlight active exploration as a key ingredient for scalable and adaptive self-evolving vision-language systems.

Figures (6)

• Figure 1: Comparison between Passive Approaches (left) and Active-Zero (right). Unlike traditional methods limited by fixed data boundaries, Active-Zero actively explores open-world environments. It employs a Searcher to retrieve high-value images, forming a closed-loop auto-curriculum with the Questioner and Solver to scale VLM reasoning.
• Figure 2: Overview of the Active-Zero framework. The system facilitates an iterative, three-stage self-play cycle to enhance VLM reasoning: (Stage 1) The Searcher ($\mathcal{S}$) is optimized to retrieve informative images from the environment by balancing a challenge reward (targeting the Solver's uncertainty) and a repetition penalty. (Stage 2) The Questioner ($\mathcal{Q}$) synthesizes complex, multi-step questions from the curated images to maximize instructional utility. (Stage 3) The Solver ($\mathcal{V}$) is trained on the generated tasks to improve reasoning accuracy and consensus via reinforcement learning. The entire process is iterative, allowing the agents to co-evolve as the Solver’s capability frontier shifts.
• Figure 3: Co-evolution of Solver and Searcher. Active-Zero shows superior accuracy scaling in (a), driven by the Searcher's ability to maximize uncertainty (b) while maintaining high sample diversity (c) through RL-based optimization.
• Figure 4: Consensus-score distribution of generated VL tasks under Active-Zero and Random Sampling across training versions (v1–v3). The consensus score measures agreement among multiple Solver rollouts on the same question.
• Figure 5: Category distribution of Searcher-retrieved images. Pie charts show the percentage of retrieved images in each task category for Active-Zero v1 and v2, with (left) and without (right) domain conditioning. Categories are identified based on source datasets in $\mathcal{D}_{env}$.