Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SWE-Universe: Scale Real-World Verifiable Environments to Millions

Mouxiang Chen, Lei Zhang, Yunlong Feng, Xuwu Wang, Wenting Zhao, Ruisheng Cao, Jiaxi Yang, Jiawei Chen, Mingze Li, Zeyao Ma, Hao Ge, Zongmeng Zhang, Zeyu Cui, Dayiheng Liu, Jingren Zhou, Jianling Sun, Junyang Lin, Binyuan Hui

TL;DR

SWE-Universe tackles the challenge of generating scalable, real-world, verifiable software engineering environments by deploying an autonomous building agent that uses iterative self-verification and in-loop hacking detection to convert issue-linked PRs into executable environments and verifiers. The framework achieves million-scale, multilingual task generation (807,693 instances) and demonstrates substantial value for both supervised and reinforcement learning, including a 75.3% SWE-Bench Verified score for Qwen3-Max-Thinking. Key contributions include a robust patch-separation pipeline, a lightweight MoE-based builder model (Qwen-Next-80A3), and a scalable infra stack (MegaFlow on Alibaba Cloud) that enables large-scale deployment and evaluation. The results show strong generalization across languages, improved mid-training benchmarks, and effective agentic RL signals, establishing SWE-Universe as a practical resource for advancing coding agents in real-world software engineering tasks.

Abstract

We propose SWE-Universe, a scalable and efficient framework for automatically constructing real-world software engineering (SWE) verifiable environments from GitHub pull requests (PRs). To overcome the prevalent challenges of automatic building, such as low production yield, weak verifiers, and prohibitive cost, our framework utilizes a building agent powered by an efficient custom-trained model. This agent employs iterative self-verification and in-loop hacking detection to ensure the reliable generation of high-fidelity, verifiable tasks. Using this method, we scale the number of real-world multilingual SWE environments to a million scale (807,693). We demonstrate the profound value of our environments through large-scale agentic mid-training and reinforcement learning. Finally, we applied this technique to Qwen3-Max-Thinking and achieved a score of 75.3% on SWE-Bench Verified. Our work provides both a critical resource and a robust methodology to advance the next generation of coding agents.

SWE-Universe: Scale Real-World Verifiable Environments to Millions

TL;DR

SWE-Universe tackles the challenge of generating scalable, real-world, verifiable software engineering environments by deploying an autonomous building agent that uses iterative self-verification and in-loop hacking detection to convert issue-linked PRs into executable environments and verifiers. The framework achieves million-scale, multilingual task generation (807,693 instances) and demonstrates substantial value for both supervised and reinforcement learning, including a 75.3% SWE-Bench Verified score for Qwen3-Max-Thinking. Key contributions include a robust patch-separation pipeline, a lightweight MoE-based builder model (Qwen-Next-80A3), and a scalable infra stack (MegaFlow on Alibaba Cloud) that enables large-scale deployment and evaluation. The results show strong generalization across languages, improved mid-training benchmarks, and effective agentic RL signals, establishing SWE-Universe as a practical resource for advancing coding agents in real-world software engineering tasks.

Abstract

We propose SWE-Universe, a scalable and efficient framework for automatically constructing real-world software engineering (SWE) verifiable environments from GitHub pull requests (PRs). To overcome the prevalent challenges of automatic building, such as low production yield, weak verifiers, and prohibitive cost, our framework utilizes a building agent powered by an efficient custom-trained model. This agent employs iterative self-verification and in-loop hacking detection to ensure the reliable generation of high-fidelity, verifiable tasks. Using this method, we scale the number of real-world multilingual SWE environments to a million scale (807,693). We demonstrate the profound value of our environments through large-scale agentic mid-training and reinforcement learning. Finally, we applied this technique to Qwen3-Max-Thinking and achieved a score of 75.3% on SWE-Bench Verified. Our work provides both a critical resource and a robust methodology to advance the next generation of coding agents.
Paper Structure (33 sections, 5 figures, 2 tables)

This paper contains 33 sections, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology: Scalability and Reliability
  3. Key Designs for SWE-Universe
  4. PR Crawling and Patch Separation.
  5. Agent-based Environment Building.
  6. Toolset.
  7. Iterative Validation.
  8. In-loop Hacking Detection.
  9. Further Analysis
  10. Case Study
  11. Quality Analysis
  12. Efficient Building and Benchmarking
  13. Model Training and Deployment
  14. Benchmark
  15. Result Analysis
  16. ...and 18 more sections

Figures (5)

  • Figure 1: A comparison of the number of instances in real-world SWE instances. Our multilingual SWE-Universe is significantly larger than other recent multilingual efforts like MiMo-V2-Flash xiao2026mimo, DeepSeek-V3.2 deepseekv32, and Multi-SWE-RL multi-swe-bench, as well as prominent Python-only datasets including SWE-rebench swe-rebench, SWE-Gym pan2024training, CWM copet2025cwm, and SWE-Bench swe-bench.
  • Figure 2: Our SWE-Universe framework for scalable and reliable environment building. The pipeline is built around a building agent that proposes a verifier (evaluation.sh). Two key components ensure quality and yield: an in-loop Hacking Detector that preemptively rejects superficial scripts, and an Iterative Validation loop where the agent self-corrects based on feedback from testing its verifier against both buggy and resolved code states.
  • Figure 3: Three types of evaluation.sh. We only accept the first two types of evaluation scripts.
  • Figure 4: Task Quality vs. Dataset Size (Log-Scale). Task quality is measured as the fraction of high-quality samples.
  • Figure 5: (a) Performance scaling trends of the Qwen3-Next-80A3 model during mid-training. (b) Reinforcement learning curve of the Qwen3-30B-A3B for SWE-Bench Multilingual.