Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem

Weixun Wang, XiaoXiao Xu, Wanhe An, Fangwen Dai, Wei Gao, Yancheng He, Ju Huang, Qiang Ji, Hanqi Jin, Xiaoyang Li, Yang Li, Zhongwen Li, Shirong Lin, Jiashun Liu, Zenan Liu, Tao Luo, Dilxat Muhtar, Yuanbin Qu, Jiaqiang Shi, Qinghui Sun, Yingshui Tan, Hao Tang, Runze Wang, Yi Wang, Zhaoguo Wang, Yanan Wu, Shaopan Xiong, Binchen Xu, Xander Xu, Yuchi Xu, Qipeng Zhang, Xixia Zhang, Haizhou Zhao, Jie Zhao, Shuaibing Zhao, Baihui Zheng, Jianhui Zheng, Suhang Zheng, Yanni Zhu, Mengze Cai, Kerui Cao, Xitong Chen, Yue Dai, Lifan Du, Tao Feng, Tao He, Jin Hu, Yijie Hu, Ziyu Jiang, Cheng Li, Xiang Li, Jing Liang, Xin Lin, Chonghuan Liu, ZhenDong Liu, Zhiqiang Lv, Haodong Mi, Yanhu Mo, Junjia Ni, Shixin Pei, Jingyu Shen, XiaoShuai Song, Cecilia Wang, Chaofan Wang, Kangyu Wang, Pei Wang, Tao Wang, Wei Wang, Ke Xiao, Mingyu Xu, Tiange Xu, Nan Ya, Siran Yang, Jianan Ye, Yaxing Zang, Duo Zhang, Junbo Zhang, Boren Zheng, Wanxi Deng, Ling Pan, Lin Qu, Wenbo Su, Jiamang Wang, Wei Wang, Hu Wei, Minggang Wu, Cheng Yu, Bing Zhao, Zhicheng Zheng, Bo Zheng

TL;DR

The paper addresses the challenge of building robust agentic LLMs capable of long-horizon interaction in real environments by introducing the Agentic Learning Ecosystem (ALE), comprising ROLL for scalable RL training, ROCK for sandboxed execution, and iFlow CLI for context management. It presents ROME, an open-source agent grounded in ALE and trained on over a million trajectories, enhanced by the Interaction-based Policy Alignment (IPA) RL algorithm that credits semantic interaction chunks rather than token-level steps. A principled data composition framework (basic code-centric data and agentic data) and a three-stage training pipeline (Continual Pre-Training, Supervised Fine-Tuning, and Agentic RL) underpin ROME’s development, with rigorous safety-oriented data curation and a comprehensive evaluation suite, including Terminal Bench Pro. Empirical results show ROME achieving strong performance on terminal and tool-use benchmarks, rivaling larger models and validating ALE’s production-ready, end-to-end pipeline. The work argues for co-design of training infrastructure, executable environments, and evaluation protocols to advance practical, scalable agentic AI in the open-source ecosystem.

Abstract

Agentic crafting requires LLMs to operate in real-world environments over multiple turns by taking actions, observing outcomes, and iteratively refining artifacts. Despite its importance, the open-source community lacks a principled, end-to-end ecosystem to streamline agent development. We introduce the Agentic Learning Ecosystem (ALE), a foundational infrastructure that optimizes the production pipeline for agent LLMs. ALE consists of three components: ROLL, a post-training framework for weight optimization; ROCK, a sandbox environment manager for trajectory generation; and iFlow CLI, an agent framework for efficient context engineering. We release ROME (ROME is Obviously an Agentic Model), an open-source agent grounded by ALE and trained on over one million trajectories. Our approach includes data composition protocols for synthesizing complex behaviors and a novel policy optimization algorithm, Interaction-based Policy Alignment (IPA), which assigns credit over semantic interaction chunks rather than individual tokens to improve long-horizon training stability. Empirically, we evaluate ROME within a structured setting and introduce Terminal Bench Pro, a benchmark with improved scale and contamination control. ROME demonstrates strong performance across benchmarks like SWE-bench Verified and Terminal Bench, proving the effectiveness of the ALE infrastructure.

Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem

TL;DR

The paper addresses the challenge of building robust agentic LLMs capable of long-horizon interaction in real environments by introducing the Agentic Learning Ecosystem (ALE), comprising ROLL for scalable RL training, ROCK for sandboxed execution, and iFlow CLI for context management. It presents ROME, an open-source agent grounded in ALE and trained on over a million trajectories, enhanced by the Interaction-based Policy Alignment (IPA) RL algorithm that credits semantic interaction chunks rather than token-level steps. A principled data composition framework (basic code-centric data and agentic data) and a three-stage training pipeline (Continual Pre-Training, Supervised Fine-Tuning, and Agentic RL) underpin ROME’s development, with rigorous safety-oriented data curation and a comprehensive evaluation suite, including Terminal Bench Pro. Empirical results show ROME achieving strong performance on terminal and tool-use benchmarks, rivaling larger models and validating ALE’s production-ready, end-to-end pipeline. The work argues for co-design of training infrastructure, executable environments, and evaluation protocols to advance practical, scalable agentic AI in the open-source ecosystem.

Abstract

Agentic crafting requires LLMs to operate in real-world environments over multiple turns by taking actions, observing outcomes, and iteratively refining artifacts. Despite its importance, the open-source community lacks a principled, end-to-end ecosystem to streamline agent development. We introduce the Agentic Learning Ecosystem (ALE), a foundational infrastructure that optimizes the production pipeline for agent LLMs. ALE consists of three components: ROLL, a post-training framework for weight optimization; ROCK, a sandbox environment manager for trajectory generation; and iFlow CLI, an agent framework for efficient context engineering. We release ROME (ROME is Obviously an Agentic Model), an open-source agent grounded by ALE and trained on over one million trajectories. Our approach includes data composition protocols for synthesizing complex behaviors and a novel policy optimization algorithm, Interaction-based Policy Alignment (IPA), which assigns credit over semantic interaction chunks rather than individual tokens to improve long-horizon training stability. Empirically, we evaluate ROME within a structured setting and introduce Terminal Bench Pro, a benchmark with improved scale and contamination control. ROME demonstrates strong performance across benchmarks like SWE-bench Verified and Terminal Bench, proving the effectiveness of the ALE infrastructure.
Paper Structure (56 sections, 10 equations, 18 figures, 8 tables)

This paper contains 56 sections, 10 equations, 18 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Agentic Learning Ecosystem: ROME Wasn't Built in a Day
  3. System Overview
  4. Agentic RL Training Framework: ROLL
  5. Agentic Training Pipeline.
  6. Fine-grained Rollout.
  7. Asynchronous Training.
  8. Train--Rollout Multiplexing.
  9. Environment Execution Engine: ROCK
  10. System Architecture and Workflow.
  11. API Interfaces.
  12. Agent Native Mode.
  13. Agent Framework: iFlow CLI
  14. The Role of iFlow CLI in Agentic Training.
  15. System Architecture and Workflow.
  16. ...and 41 more sections

Figures (18)

  • Figure 1: Overview of the Agentic Learning Ecosystem (ALE) and ROME Performance.
  • Figure 2: The overview of agentic RL ecosystem (a) and its training pipeline (b).
  • Figure 3: ROLL Architecture. (a) ROLL pipelines LLM generation, environment interaction, and reward phases at trajectory-level granularity. Training is also decoupled via a sample buffer using an asynchronous ratio to manage staleness. (b) ROLL multiplexes a dynamic GPU pool by shrinking rollout resources for bursty training and expanding them back during demand peaks.
  • Figure 4: ROCK System Architecture.
  • Figure 5: The overview of iFlow CLI architecture and execution.
  • ...and 13 more figures