BricksRL: A Platform for Democratizing Robotics and Reinforcement Learning Research and Education with LEGO

TL;DR

BricksRL introduces a LEGO-based platform that democratizes robotics and reinforcement learning by unifying Pybricks with TorchRL to train real-world robots through gym-like environments. It presents a modular architecture (agent, environment, robot) and demonstrates three LEGO robots (2Wheeler, Walker, RoboArm) across multiple tasks, including online and offline RL and sim2real transfer. The work highlights extensibility with sensors (e.g., cameras), provides building plans and datasets, and reports training on a standard laptop, often under 120 minutes, making advanced RL accessible to researchers, educators, and hobbyists. Overall, BricksRL establishes a practical, open-source foundation for affordable, reusable robotic learning, with future directions toward more complex robots, multi-agent settings, and transformer-based imitation learning.

Abstract

We present BricksRL, a platform designed to democratize access to robotics for reinforcement learning research and education. BricksRL facilitates the creation, design, and training of custom LEGO robots in the real world by interfacing them with the TorchRL library for reinforcement learning agents. The integration of TorchRL with the LEGO hubs, via Bluetooth bidirectional communication, enables state-of-the-art reinforcement learning training on GPUs for a wide variety of LEGO builds. This offers a flexible and cost-efficient approach for scaling and also provides a robust infrastructure for robot-environment-algorithm communication. We present various experiments across tasks and robot configurations, providing built plans and training results. Furthermore, we demonstrate that inexpensive LEGO robots can be trained end-to-end in the real world to achieve simple tasks, with training times typically under 120 minutes on a normal laptop. Moreover, we show how users can extend the capabilities, exemplified by the successful integration of non-LEGO sensors. By enhancing accessibility to both robotics and reinforcement learning, BricksRL establishes a strong foundation for democratized robotic learning in research and educational settings.

Figures (8)

• Figure 1: Communication overview of the agent, environment and robot.
• Figure 2: Three robots that we used in the experiments: (a) 2Wheeler, (b) Walker, (c) RoboArm.
• Figure 3: Training results for 2Wheeler robot for the RunAway-v0 and the Spinning-v0 environment.
• Figure 4: Training performance for Walker robot for the Walker-v0 and the WalkerSim-v0 environment.
• Figure 5: Training outcomes for the RoboArm robot in both the RoboArm-v0 and RoboArmSim-v0 environments. The plot also includes the final error at the epoch's last step and the total number of episode steps.