Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RSL-RL: A Learning Library for Robotics Research

Clemens Schwarke, Mayank Mittal, Nikita Rudin, David Hoeller, Marco Hutter

TL;DR

RSL-RL tackles the need for a lightweight, robotics-focused RL library that is easy to modify and extend. It centers on a minimalist architecture (Runners, Algorithms, Networks), robotics-first algorithms (PPO and DAgger-style BC), and auxiliary techniques (symmetry augmentation and curiosity-based exploration) optimized for GPU-based, large-scale simulation. The framework provides logging utilities and distributed training, implemented in PyTorch with a VecEnv/TensorDict interface to streamline robotics experiments. Its adoption across legged locomotion, sim-to-real, and multi-skill control demonstrates practical impact in real-world robotics research.

Abstract

RSL-RL is an open-source Reinforcement Learning library tailored to the specific needs of the robotics community. Unlike broad general-purpose frameworks, its design philosophy prioritizes a compact and easily modifiable codebase, allowing researchers to adapt and extend algorithms with minimal overhead. The library focuses on algorithms most widely adopted in robotics, together with auxiliary techniques that address robotics-specific challenges. Optimized for GPU-only training, RSL-RL achieves high-throughput performance in large-scale simulation environments. Its effectiveness has been validated in both simulation benchmarks and in real-world robotic experiments, demonstrating its utility as a lightweight, extensible, and practical framework to develop learning-based robotic controllers. The library is open-sourced at: https://github.com/leggedrobotics/rsl_rl.

RSL-RL: A Learning Library for Robotics Research

TL;DR

RSL-RL tackles the need for a lightweight, robotics-focused RL library that is easy to modify and extend. It centers on a minimalist architecture (Runners, Algorithms, Networks), robotics-first algorithms (PPO and DAgger-style BC), and auxiliary techniques (symmetry augmentation and curiosity-based exploration) optimized for GPU-based, large-scale simulation. The framework provides logging utilities and distributed training, implemented in PyTorch with a VecEnv/TensorDict interface to streamline robotics experiments. Its adoption across legged locomotion, sim-to-real, and multi-skill control demonstrates practical impact in real-world robotics research.

Abstract

RSL-RL is an open-source Reinforcement Learning library tailored to the specific needs of the robotics community. Unlike broad general-purpose frameworks, its design philosophy prioritizes a compact and easily modifiable codebase, allowing researchers to adapt and extend algorithms with minimal overhead. The library focuses on algorithms most widely adopted in robotics, together with auxiliary techniques that address robotics-specific challenges. Optimized for GPU-only training, RSL-RL achieves high-throughput performance in large-scale simulation environments. Its effectiveness has been validated in both simulation benchmarks and in real-world robotic experiments, demonstrating its utility as a lightweight, extensible, and practical framework to develop learning-based robotic controllers. The library is open-sourced at: https://github.com/leggedrobotics/rsl_rl.

Paper Structure

This paper contains 8 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Features
  3. Algorithms
  4. Auxiliary Techniques
  5. Utilities
  6. Implementation Details
  7. Applications in Research
  8. When and When Not to Use RSL-RL

Figures (1)

  • Figure 1: Overview of the framework.RSL-RL consists of three main components: Runners, Algorithms, and Networks, which can be easily modified independently. The framework comes with support for common logging choices and useful extensions for robotics.