Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Meta-World+: An Improved, Standardized, RL Benchmark

Reginald McLean, Evangelos Chatzaroulas, Luc McCutcheon, Frank Röder, Tianhe Yu, Zhanpeng He, K. R. Zentner, Ryan Julian, J K Terry, Isaac Woungang, Nariman Farsad, Pablo Samuel Castro

TL;DR

The paper addresses inconsistent versioning in the Meta-World benchmark that impeded fair evaluation of multi-task and meta-RL algorithms. It presents a preserved, reproducible reimplementation with selectable reward versions (V1/V2), new task sets (MT25/ML25), and Gymnasium integration, together with empirical analyses across algorithms and reward schemes. Key contributions include exposing cross-version inconsistencies, enabling customizable task sets, and detailing a scalable, open-source benchmark design intended to guide future multi-task/meta-RL benchmarking. The work aims to standardize evaluation practices, improve reproducibility, and inform benchmark design for more robust assessment of algorithmic advances.

Abstract

Meta-World is widely used for evaluating multi-task and meta-reinforcement learning agents, which are challenged to master diverse skills simultaneously. Since its introduction however, there have been numerous undocumented changes which inhibit a fair comparison of algorithms. This work strives to disambiguate these results from the literature, while also leveraging the past versions of Meta-World to provide insights into multi-task and meta-reinforcement learning benchmark design. Through this process we release a new open-source version of Meta-World (https://github.com/Farama-Foundation/Metaworld/) that has full reproducibility of past results, is more technically ergonomic, and gives users more control over the tasks that are included in a task set.

Meta-World+: An Improved, Standardized, RL Benchmark

TL;DR

The paper addresses inconsistent versioning in the Meta-World benchmark that impeded fair evaluation of multi-task and meta-RL algorithms. It presents a preserved, reproducible reimplementation with selectable reward versions (V1/V2), new task sets (MT25/ML25), and Gymnasium integration, together with empirical analyses across algorithms and reward schemes. Key contributions include exposing cross-version inconsistencies, enabling customizable task sets, and detailing a scalable, open-source benchmark design intended to guide future multi-task/meta-RL benchmarking. The work aims to standardize evaluation practices, improve reproducibility, and inform benchmark design for more robust assessment of algorithmic advances.

Abstract

Meta-World is widely used for evaluating multi-task and meta-reinforcement learning agents, which are challenged to master diverse skills simultaneously. Since its introduction however, there have been numerous undocumented changes which inhibit a fair comparison of algorithms. This work strives to disambiguate these results from the literature, while also leveraging the past versions of Meta-World to provide insights into multi-task and meta-reinforcement learning benchmark design. Through this process we release a new open-source version of Meta-World (https://github.com/Farama-Foundation/Metaworld/) that has full reproducibility of past results, is more technically ergonomic, and gives users more control over the tasks that are included in a task set.
Paper Structure (29 sections, 9 figures, 10 tables)

This paper contains 29 sections, 9 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. RL Problem Statements
  4. Meta-World
  5. Task Sets
  6. Reward Functions
  7. Empirical Results
  8. Multi-task RL Results
  9. Meta-RL Results
  10. Results over various task set sizes
  11. Overall Benchmark Improvements
  12. Reasoning for New Version of Meta-World
  13. Gymnasium Integration
  14. Conclusion
  15. Meta-World
  16. ...and 14 more sections

Figures (9)

  • Figure 1: (a) Per-timestep rewards for the pick-place task from Meta-World. (Left) The Y-axis shows the scale of the V1 rewards, while the right Y-axis shows the scale of the V2 rewards. The right plot shows the Q-Values per-update batch through training on MT10 (log scale). (b) Q-Function loss and mean success rate of the MTMHSAC on MT10. The left y-axis is the Q-Function loss (log scale), while the right y-axis is the mean success rate. Both blue plots are the V1 rewards, while both red plots are the V2 rewards. Plots with circular points are Q-Function loss, while plots with X points are success rates (with 95% CIs).
  • Figure 2: Effects of reward functions on selected multi-task algorithms on (a) MT10 and (b) MT50.
  • Figure 3: IQM results over (a) ML10 and (b) ML45. Plotted results are the testing success rates.
  • Figure 4: (a) Effects of various task set sizes on the performance of the MTMHSAC agent. (b) Effects of various task set sizes on the performance of the MAML agent.
  • Figure 5: Process for using Meta-World in previous versions (left), and our updated version (right).
  • ...and 4 more figures