Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Data

Zongxia Li, Hongyang Du, Chengsong Huang, Xiyang Wu, Lantao Yu, Yicheng He, Jing Xie, Xiaomin Wu, Zhichao Liu, Jiarui Zhang, Fuxiao Liu

TL;DR

Multi-model Multimodal Zero (MM-Zero), the first RL-based framework to achieve zero-data self-evolution for VLM reasoning, establishes a scalable path toward self-evolving multi-model systems for multimodal models, extending the frontier of self-improvement beyond the conventional two-model paradigm.

Abstract

Self-evolving has emerged as a key paradigm for improving foundational models such as Large Language Models (LLMs) and Vision Language Models (VLMs) with minimal human intervention. While recent approaches have demonstrated that LLM agents can self-evolve from scratch with little to no data, VLMs introduce an additional visual modality that typically requires at least some seed data, such as images, to bootstrap the self-evolution process. In this work, we present Multi-model Multimodal Zero (MM-Zero), the first RL-based framework to achieve zero-data self-evolution for VLM reasoning. Moving beyond prior dual-role (Proposer and Solver) setups, MM-Zero introduces a multi-role self-evolving training framework comprising three specialized roles: a Proposer that generates abstract visual concepts and formulates questions; a Coder that translates these concepts into executable code (e.g., Python, SVG) to render visual images; and a Solver that performs multimodal reasoning over the generated visual content. All three roles are initialized from the same base model and trained using Group Relative Policy Optimization (GRPO), with carefully designed reward mechanisms that integrate execution feedback, visual verification, and difficulty balancing. Our experiments show that MM-Zero improves VLM reasoning performance across a wide range of multimodal benchmarks. MM-Zero establishes a scalable path toward self-evolving multi-model systems for multimodal models, extending the frontier of self-improvement beyond the conventional two-model paradigm.

MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Data

TL;DR

Multi-model Multimodal Zero (MM-Zero), the first RL-based framework to achieve zero-data self-evolution for VLM reasoning, establishes a scalable path toward self-evolving multi-model systems for multimodal models, extending the frontier of self-improvement beyond the conventional two-model paradigm.

Abstract

Self-evolving has emerged as a key paradigm for improving foundational models such as Large Language Models (LLMs) and Vision Language Models (VLMs) with minimal human intervention. While recent approaches have demonstrated that LLM agents can self-evolve from scratch with little to no data, VLMs introduce an additional visual modality that typically requires at least some seed data, such as images, to bootstrap the self-evolution process. In this work, we present Multi-model Multimodal Zero (MM-Zero), the first RL-based framework to achieve zero-data self-evolution for VLM reasoning. Moving beyond prior dual-role (Proposer and Solver) setups, MM-Zero introduces a multi-role self-evolving training framework comprising three specialized roles: a Proposer that generates abstract visual concepts and formulates questions; a Coder that translates these concepts into executable code (e.g., Python, SVG) to render visual images; and a Solver that performs multimodal reasoning over the generated visual content. All three roles are initialized from the same base model and trained using Group Relative Policy Optimization (GRPO), with carefully designed reward mechanisms that integrate execution feedback, visual verification, and difficulty balancing. Our experiments show that MM-Zero improves VLM reasoning performance across a wide range of multimodal benchmarks. MM-Zero establishes a scalable path toward self-evolving multi-model systems for multimodal models, extending the frontier of self-improvement beyond the conventional two-model paradigm.
Paper Structure (48 sections, 14 equations, 3 figures, 5 tables)

This paper contains 48 sections, 14 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Preliminaries
  4. Training Pipeline
  5. Proposer.
  6. Coder.
  7. Solver.
  8. Training data filtering.
  9. Reward Formulations
  10. Proposer Reward Formulation
  11. 1. Code Generation & Visual Rendering ($\mathds{1}_{\text{exec}}(C_i)$).
  12. 2. Solvability Score ($R_{\text{solv}}$).
  13. 3. Difficulty Score ($R_{\text{diff}}$).
  14. 4. Easy-Hard Penalty ($r_{\text{eh}}$).
  15. 5. Content-Type Diversity Penalty ($r_{\text{ct}}$).
  16. ...and 33 more sections

Figures (3)

  • Figure 1: Overview of the MM-Zero self-evolving framework. The Proposer generates a quadruple (textual description, easy question, easy answer, hard question). The Coder converts the description into executable code to render a figure. The Solver first answers the easy question to verify semantic correctness and provide reward signals to update the Coder, then answers the hard question using majority voting to generate pseudo-labels for its own training while providing a difficulty reward to optimize the Proposer.
  • Figure 2: Throughout training, the coder's image rendering success rate steadily improves, indicating stronger code generation ability. Additionally, image solvability also increases, meaning the rendered images become more faithful to the original and contain sufficient information to answer the question.
  • Figure 3: Evolution of the proposer and coder. This figure presents examples of questions generated by the self-evolving Vision-Language model. As the training steps progress, the visual context becomes more intricate, and the model generates increasingly challenging questions. While the upper bound of complexity grows, foundational and simple questions are still preserved. The questions are formulated by Qwen3-VL-8B-Instruct.