INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learning

TL;DR

INTELLECT-2 demonstrates a pioneering globally distributed reinforcement learning setup for a 32B language model, leveraging asynchronous two-step GRPO training across permissionless compute. The paper introduces PRIME-RL for distributed RL, TOPLOC for trustless inference, SHARDCAST for efficient weight broadcasting, and the Prime Intellect Protocol to coordinate a decentralized compute ecosystem, achieving notable compute overlap and improved math/coding reasoning over prior 32B baselines. Empirical results show feasible throughput, scalable inference-heavy workloads, and a favorable shift toward inference-dominated compute in decentralized regimes, with open-source release of data and tooling. The work lays groundwork for future directions including increased inference-to-training ratios, tool-assisted reasoning, crowdsourced RL environments, and model merging via DiLoCo, advancing open, scalable decentralized AI research.

Abstract

We introduce INTELLECT-2, the first globally distributed reinforcement learning (RL) training run of a 32 billion parameter language model. Unlike traditional centralized training efforts, INTELLECT-2 trains a reasoning model using fully asynchronous RL across a dynamic, heterogeneous swarm of permissionless compute contributors. To enable a training run with this unique infrastructure, we built various components from scratch: we introduce PRIME-RL, our training framework purpose-built for distributed asynchronous reinforcement learning, based on top of novel components such as TOPLOC, which verifies rollouts from untrusted inference workers, and SHARDCAST, which efficiently broadcasts policy weights from training nodes to inference workers. Beyond infrastructure components, we propose modifications to the standard GRPO training recipe and data filtering techniques that were crucial to achieve training stability and ensure that our model successfully learned its training objective, thus improving upon QwQ-32B, the state of the art reasoning model in the 32B parameter range. We open-source INTELLECT-2 along with all of our code and data, hoping to encourage and enable more open research in the field of decentralized training.

Figures (13)

• Figure 1: System Overview of intellect-2 Distributed RL Training Infrastructure.
• Figure 2: Overview of the Shardcast policy weight distribution network.
• Figure 3: An illustration of toploc. The Inference Provider performs batched inferences and generates commits for the computations performed, while the Verifier audits these commits up to $100\times$ faster than the time it takes the inference provider to generate the responses. Based on the verification outcome, rewards are granted for valid batches and penalties are applied for invalid ones. Further speedup can be obtained for the Verifier by not checking every batch but instead sampling randomly. Since the Inference Provider does not know which generations will be checked by the Verifier, they are incentivized to be honest on all generations to collect the reward and avoid receiving the penalty.
• Figure 4: Overview of Protocol Testnet Infrastructure.
• Figure 5: Overview of toploc Validator Setup.