Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark

Jiaming Ji, Borong Zhang, Jiayi Zhou, Xuehai Pan, Weidong Huang, Ruiyang Sun, Yiran Geng, Yifan Zhong, Juntao Dai, Yaodong Yang

TL;DR

An environment suite called Safety-Gymnasium, which encompasses safety-critical tasks in both single and multi-agent scenarios, accepting vector and vision-only input, and a library of algorithms named Safe Policy Optimization (SafePO), comprising 16 state-of-the-art SafeRL algorithms, which can serve as a validation tool for the research community.

Abstract

Artificial intelligence (AI) systems possess significant potential to drive societal progress. However, their deployment often faces obstacles due to substantial safety concerns. Safe reinforcement learning (SafeRL) emerges as a solution to optimize policies while simultaneously adhering to multiple constraints, thereby addressing the challenge of integrating reinforcement learning in safety-critical scenarios. In this paper, we present an environment suite called Safety-Gymnasium, which encompasses safety-critical tasks in both single and multi-agent scenarios, accepting vector and vision-only input. Additionally, we offer a library of algorithms named Safe Policy Optimization (SafePO), comprising 16 state-of-the-art SafeRL algorithms. This comprehensive library can serve as a validation tool for the research community. By introducing this benchmark, we aim to facilitate the evaluation and comparison of safety performance, thus fostering the development of reinforcement learning for safer, more reliable, and responsible real-world applications. The website of this project can be accessed at https://sites.google.com/view/safety-gymnasium.

Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark

TL;DR

An environment suite called Safety-Gymnasium, which encompasses safety-critical tasks in both single and multi-agent scenarios, accepting vector and vision-only input, and a library of algorithms named Safe Policy Optimization (SafePO), comprising 16 state-of-the-art SafeRL algorithms, which can serve as a validation tool for the research community.

Abstract

Artificial intelligence (AI) systems possess significant potential to drive societal progress. However, their deployment often faces obstacles due to substantial safety concerns. Safe reinforcement learning (SafeRL) emerges as a solution to optimize policies while simultaneously adhering to multiple constraints, thereby addressing the challenge of integrating reinforcement learning in safety-critical scenarios. In this paper, we present an environment suite called Safety-Gymnasium, which encompasses safety-critical tasks in both single and multi-agent scenarios, accepting vector and vision-only input. Additionally, we offer a library of algorithms named Safe Policy Optimization (SafePO), comprising 16 state-of-the-art SafeRL algorithms. This comprehensive library can serve as a validation tool for the research community. By introducing this benchmark, we aim to facilitate the evaluation and comparison of safety performance, thus fostering the development of reinforcement learning for safer, more reliable, and responsible real-world applications. The website of this project can be accessed at https://sites.google.com/view/safety-gymnasium.
Paper Structure (35 sections, 5 equations, 23 figures, 32 tables)

This paper contains 35 sections, 5 equations, 23 figures, 32 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Safety Environments
  4. SafeRL Algorithms
  5. Preliminaries
  6. Constrained Markov decision process
  7. Constrained Markov Game
  8. Safety Environments: Safety-Gymnasium
  9. Gynasium-based Learning Environments
  10. Supported Robots
  11. Supported Tasks
  12. Supported Constraints
  13. Vision-only tasks
  14. Issac-Gym-based Learning Environments
  15. Safe Policy Optimization Algorithms: SafePO
  16. ...and 20 more sections

Figures (23)

  • Figure 1: Using Safety-Gymnasium to create, step, render a specific safety-task environment.
  • Figure 2: Upper: The Single-Agent Robots of Gymnasium-based Environments. Lower: The Multi-Agent Robots of Gymnasium-based Environments.
  • Figure 3: Upper: Tasks of Gymnasium-based Environments; Lower: Constraints of Gymnasium-based Environments.
  • Figure 4: Vision-only Tasks of Gymnasium-based Environments.
  • Figure 5: The RGB and RGB-D input of Gymnasium-based Environments.
  • ...and 18 more figures