Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learning

Kun Lei, Huanyu Li, Dongjie Yu, Zhenyu Wei, Lingxiao Guo, Zhennan Jiang, Ziyu Wang, Shiyu Liang, Huazhe Xu

TL;DR

RL-100 presents a deployment-focused real-world RL framework that builds on diffusion-based visuomotor policies to unify imitation learning and RL via a single PPO-style objective across denoising steps. It combines imitation learning, iterative offline RL with data expansion, and targeted online RL, followed by distillation to a one-step consistency policy for high-frequency control. The approach demonstrates 100% task success across seven real-world manipulation tasks, strong zero-shot and few-shot generalization, and robust performance under disturbances and in a mall deployment. The results highlight the practical viability of starting from human priors, aligning with human-grounded metrics, and iteratively self-improving toward deployment-grade reliability and efficiency. This work suggests a scalable path to real-world robot learning with diffusion policies and consistency distillation, enabling rapid transfer to homes and factories while maintaining safety and robustness.

Abstract

Real-world robotic manipulation in homes and factories demands reliability, efficiency, and robustness that approach or surpass the performance of skilled human operators. We present RL-100, a real-world reinforcement learning framework built on diffusion-based visuomotor policies. RL-100 unifies imitation and reinforcement learning under a single PPO-style objective applied within the denoising process, yielding conservative and stable policy improvements across both offline and online stages. To meet deployment latency constraints, we employ a lightweight consistency distillation procedure that compresses multi-step diffusion into a one-step controller for high-frequency control. The framework is task-, embodiment-, and representation-agnostic, and supports both single-action outputs and action-chunking control. We evaluate RL-100 on seven diverse real-robot manipulation tasks, ranging from dynamic pushing and agile bowling to pouring, cloth folding, unscrewing, and multi-stage juicing. RL-100 attains 100% success across evaluated trials, achieving 900 out of 900 successful episodes, including up to 250 out of 250 consecutive trials on one task, and matches or surpasses expert teleoperators in time-to-completion. Without retraining, a single policy attains approximately 90% zero-shot success under environmental and dynamics shifts, adapts in a few-shot regime to significant task variations (86.7%), and remains robust to aggressive human perturbations (about 95%). In a public shopping-mall deployment, the juicing robot served random customers continuously for roughly seven hours without failure. Together, these results suggest a practical path toward deployment-ready robot learning: start from human priors, align training objectives with human-grounded metrics, and reliably extend performance beyond human demonstrations.

RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learning

TL;DR

RL-100 presents a deployment-focused real-world RL framework that builds on diffusion-based visuomotor policies to unify imitation learning and RL via a single PPO-style objective across denoising steps. It combines imitation learning, iterative offline RL with data expansion, and targeted online RL, followed by distillation to a one-step consistency policy for high-frequency control. The approach demonstrates 100% task success across seven real-world manipulation tasks, strong zero-shot and few-shot generalization, and robust performance under disturbances and in a mall deployment. The results highlight the practical viability of starting from human priors, aligning with human-grounded metrics, and iteratively self-improving toward deployment-grade reliability and efficiency. This work suggests a scalable path to real-world robot learning with diffusion policies and consistency distillation, enabling rapid transfer to homes and factories while maintaining safety and robustness.

Abstract

Real-world robotic manipulation in homes and factories demands reliability, efficiency, and robustness that approach or surpass the performance of skilled human operators. We present RL-100, a real-world reinforcement learning framework built on diffusion-based visuomotor policies. RL-100 unifies imitation and reinforcement learning under a single PPO-style objective applied within the denoising process, yielding conservative and stable policy improvements across both offline and online stages. To meet deployment latency constraints, we employ a lightweight consistency distillation procedure that compresses multi-step diffusion into a one-step controller for high-frequency control. The framework is task-, embodiment-, and representation-agnostic, and supports both single-action outputs and action-chunking control. We evaluate RL-100 on seven diverse real-robot manipulation tasks, ranging from dynamic pushing and agile bowling to pouring, cloth folding, unscrewing, and multi-stage juicing. RL-100 attains 100% success across evaluated trials, achieving 900 out of 900 successful episodes, including up to 250 out of 250 consecutive trials on one task, and matches or surpasses expert teleoperators in time-to-completion. Without retraining, a single policy attains approximately 90% zero-shot success under environmental and dynamics shifts, adapts in a few-shot regime to significant task variations (86.7%), and remains robust to aggressive human perturbations (about 95%). In a public shopping-mall deployment, the juicing robot served random customers continuously for roughly seven hours without failure. Together, these results suggest a practical path toward deployment-ready robot learning: start from human priors, align training objectives with human-grounded metrics, and reliably extend performance beyond human demonstrations.
Paper Structure (66 sections, 32 equations, 16 figures, 7 tables, 1 algorithm)

This paper contains 66 sections, 32 equations, 16 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Reinforcement Learning
  4. Offline-to-online RL.
  5. Diffusion models
  6. DDIM sampling with stochastic form
  7. Policy perspective and log-likelihood.
  8. Notation and scheduling conventions.
  9. Consistency Models
  10. Diffusion policy and RL fine-tuning
  11. RL formulation.
  12. Methods
  13. Imitation Learning
  14. Conditioning and prediction horizon.
  15. Diffusion parameterization.
  16. ...and 51 more sections

Figures (16)

  • Figure 1: Teaser. Real-robot snapshots illustrating the diversity of our task suite. Panels are ordered top-left$\rightarrow$bottom-right: (a) Dynamic Push-T; (b) Agile Bowling; (c) Pouring. (d) Dynamic Unscrewing with a dexterous hand; (e) Dual-arm Folding; (f) Juicing—stage 1 and stage 2 are shown side-by-side in the same panel; The grid highlights all seven tasks; juicing stage 1/2 are grouped into one panel for space.
  • Figure 2: Overview of RL-100. We first learn from human demonstrations through imitation learning with diffusion policies, then apply iterative offline RL with data expansion, followed by online fine-tuning for final performance optimization.
  • Figure 3: Illustrations from rollouts on seven real-world tasks. Each row shows one task, and columns are time-ordered frames (left$\rightarrow$right) subsampled from a single trajectory. From top to bottom: (a) Dynamic Push-T, (b) Agile bowling, (c) Pouring, (d) Dynamic Unscrewing, (e) Soft-towel Folding, (f) Orange Juicing -- Placing, (f) Orange Juicing -- Removal. The suite spans fluids or particles, tool-using, deformable objects manipulation, dynamic non-prehensile manipulation, and precise insertion, highlighting the diversity and dynamics of our benchmark.
  • Figure 4: Point clouds trajectories for seven tasks (ordered top-to-bottom): (A) Push-T, (B) Bowling, (C) Pouring, (D) Unscrewing, (E) Folding, (F) Orange Juicing -- Placing, (G) Orange Juicing -- Removal.
  • Figure 5: The object initialization workspaces for the seven real-world tasks. At the beginning of each episode, objects-of-interest are randomly positioned within the areas denoted by the red boundaries and annotations.
  • ...and 11 more figures