Generative Visual Code Mobile World Models

TL;DR

The paper introduces gWorld, an open-weight visual mobile GUI world model that predicts next GUI states as renderable web code, combining the text-precision of language models with the fidelity of code-driven visuals. It addresses the limitations of pixel-based and text-only WMs by using a three-step data-generation pipeline that repurposes policy trajectories, relabels states as code, and adds free-lookahead reasoning traces. A new benchmark suite, MWMBench, evaluates models on native visual states across ID and two OOD splits, showing that gWorld outperforms significantly larger baselines in both accuracy and robustness. The work demonstrates meaningful policy gains when integrating the WM with downstream agents and shows scalable data growth follows predictable power laws, suggesting substantial future performance gains with larger datasets and further architectural refinements. Overall, gWorld establishes a practical, scalable, open-weight approach to mobile GUI world modeling with strong implications for real-world GUI agents and synthetic data scaling for RL.

Abstract

Mobile Graphical User Interface (GUI) World Models (WMs) offer a promising path for improving mobile GUI agent performance at train- and inference-time. However, current approaches face a critical trade-off: text-based WMs sacrifice visual fidelity, while the inability of visual WMs in precise text rendering led to their reliance on slow, complex pipelines dependent on numerous external models. We propose a novel paradigm: visual world modeling via renderable code generation, where a single Vision-Language Model (VLM) predicts the next GUI state as executable web code that renders to pixels, rather than generating pixels directly. This combines the strengths of both approaches: VLMs retain their linguistic priors for precise text rendering while their pre-training on structured web code enables high-fidelity visual generation. We introduce gWorld (8B, 32B), the first open-weight visual mobile GUI WMs built on this paradigm, along with a data generation framework (gWorld) that automatically synthesizes code-based training data. In extensive evaluation across 4 in- and 2 out-of-distribution benchmarks, gWorld sets a new pareto frontier in accuracy versus model size, outperforming 8 frontier open-weight models over 50.25x larger. Further analyses show that (1) scaling training data via gWorld yields meaningful gains, (2) each component of our pipeline improves data quality, and (3) stronger world modeling improves downstream mobile GUI policy performance.

Figures (21)

• Figure 1: Average Instruction Accuracy (IAcc.) across all six benchmarks.gWorld 8B and 32B achieve a new pareto frontier in terms of model size (log$_{10}$ scaled). The existing pareto frontier was defined by Qwen3 VL 8B, 32B, and GLM 4.6V 106B. Notably, extremely large models (e.g., Llama 4 402B) do not reach this pareto frontier, while text-image-to-image models (e.g., Emu3.5 34B) struggle with mobile GUI dynamics.
• Figure 2: Mobile GUI world modeling via renderable code. Given an image state $S_t$ and action $A_t$, the model predicts the next state $S_{t+1}$. Our model, gWorld, generates renderable web code to ensure pixel-perfect text and structurally accurate layouts. In contrast, image-gen baselines (e.g., Qwen-Image-Edit 20B) struggle with the discrete nature of GUIs, frequently producing illegible text and distorted layouts. See Appendix Fig. \ref{['fig:qual1']}, \ref{['fig:qual2']}, \ref{['fig:qual3']} for additional qualitative examples.
• Figure 3: Schematic diagram of our data generation pipeline. We construct VLM world modeling data via three steps: (1) Repurposing offline policy trajectories into transition triplets; (2) Cross-modal relabeling of the ground-truth next state from pixels ($S^\text{image}_{t+1}$) to renderable web code ($S_{t+1}^{code}$); and (3) Synthesizing reasoning traces ($R_t$) using look-ahead access to the target state. The final training objective is to predict both the reasoning trace and the code-based next state: $(S_t, A_t) \rightarrow ({\color{darkorange}R_t, S_{t+1}^\text{code}})$. For visual succintness we denote $S_t^\text{image}$ as $S_t$ without the superscript in the diagram.
• Figure 4: Correlation between input-output similarity and model performance. Top: Pearson correlation $\rho$ between Sim($S_t$, $S_{t+1}$) and Sim($\hat{S}_{t+1}$, $S_{t+1}$). Image generation models show strong positive correlations ($\rho > 0.7$), suggesting output quality largely depends on how similar $S_t$ and $S_{t+1}$ already are. Bottom: Sim($\hat{S}_{t+1}$, $S_{t+1}$) $-$ Sim($S_t$, $S_{t+1}$) vs. Sim($S_t$, $S_{t+1}$), with the gray line indicating the score ceiling. Emu3.5 34B clusters near zero, implying Sim($\hat{S}_{t+1}$, $S_{t+1}$) $\approx$ Sim($S_t$, $S_{t+1}$); i.e., outputs nearly identical to inputs, $S_t \approx \hat{S}_{t+1}$. In contrast, gWorld 32B shows a wide vertical spread, indicating active state transformation with many samples achieving large positive gains toward the ceiling. Same analysis with Qwen-Image-Edit 20B is available in Appendix Fig. \ref{['fig:pixel_based_analysis_qwen']} with equivalent results.
• Figure 5: Data scaling laws for mobile world modeling at 8B. We fit power-law curves ($y=ax^b$) to the test performance across five distinct benchmarks as a function of training dataset size. The high coefficients of determination ($R^2 \ge 0.94$ for most splits) indicate a predictable and non-saturating relationship between data scale and performance. This suggests that our data generation pipeline has not yet reached its upper bound and will continue to improve with larger-scale repurposed trajectories.