Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NovelGym: A Flexible Ecosystem for Hybrid Planning and Learning Agents Designed for Open Worlds

Shivam Goel, Yichen Wei, Panagiotis Lymperopoulos, Klara Chura, Matthias Scheutz, Jivko Sinapov

TL;DR

NovelGym presents a modular gridworld platform for evaluating novelty-aware agents in open worlds, integrating both planning and learning paradigms. It formalizes environment and agent definitions, and introduces a novelty transform framework that can alter layout, entities, recipes, actions, dynamics, or costs in a composable way. The framework supports symbolic planning via PDDL, reinforcement learning via MDPs, and neurosymbolic hybrids, with a dedicated novelty injector and standardized evaluation Protocol and Metrics. Empirical results on a Pogostick task across five novelties demonstrate how different architectures respond to open-world changes, highlighting the value of hybrid approaches and transfer learning for rapid adaptation. The work establishes NovelGym as a flexible benchmark for open-world AI research with practical implications for developing robust, novelty-aware agents in real-world scenarios.

Abstract

As AI agents leave the lab and venture into the real world as autonomous vehicles, delivery robots, and cooking robots, it is increasingly necessary to design and comprehensively evaluate algorithms that tackle the ``open-world''. To this end, we introduce NovelGym, a flexible and adaptable ecosystem designed to simulate gridworld environments, serving as a robust platform for benchmarking reinforcement learning (RL) and hybrid planning and learning agents in open-world contexts. The modular architecture of NovelGym facilitates rapid creation and modification of task environments, including multi-agent scenarios, with multiple environment transformations, thus providing a dynamic testbed for researchers to develop open-world AI agents.

NovelGym: A Flexible Ecosystem for Hybrid Planning and Learning Agents Designed for Open Worlds

TL;DR

NovelGym presents a modular gridworld platform for evaluating novelty-aware agents in open worlds, integrating both planning and learning paradigms. It formalizes environment and agent definitions, and introduces a novelty transform framework that can alter layout, entities, recipes, actions, dynamics, or costs in a composable way. The framework supports symbolic planning via PDDL, reinforcement learning via MDPs, and neurosymbolic hybrids, with a dedicated novelty injector and standardized evaluation Protocol and Metrics. Empirical results on a Pogostick task across five novelties demonstrate how different architectures respond to open-world changes, highlighting the value of hybrid approaches and transfer learning for rapid adaptation. The work establishes NovelGym as a flexible benchmark for open-world AI research with practical implications for developing robust, novelty-aware agents in real-world scenarios.

Abstract

As AI agents leave the lab and venture into the real world as autonomous vehicles, delivery robots, and cooking robots, it is increasingly necessary to design and comprehensively evaluate algorithms that tackle the ``open-world''. To this end, we introduce NovelGym, a flexible and adaptable ecosystem designed to simulate gridworld environments, serving as a robust platform for benchmarking reinforcement learning (RL) and hybrid planning and learning agents in open-world contexts. The modular architecture of NovelGym facilitates rapid creation and modification of task environments, including multi-agent scenarios, with multiple environment transformations, thus providing a dynamic testbed for researchers to develop open-world AI agents.
Paper Structure (68 sections, 1 equation, 5 figures, 7 tables, 1 algorithm)

This paper contains 68 sections, 1 equation, 5 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Theoretical Framework
  4. Running example
  5. Environment
  6. Grid
  7. Entities
  8. Recipes
  9. States
  10. Primitive Actions
  11. Transition Dynamics
  12. Cost function
  13. Agent
  14. Symbolic Planning
  15. Reinforcement Learning
  16. ...and 53 more sections

Figures (5)

  • Figure 1: NovelGym environment representation. The figure shows a gridworld environment with various entities, as described in the legend. The red box highlights the novelty in the environment.
  • Figure 2: Illustration of the sensor representation of the agent in the environment. (Left) shows a LiDAR representation. (Right) shows an image based local view representation.
  • Figure 3: System design of the NovelGym ecosystem. Blue highlights the environment modules, and purple highlights the agent modules.
  • Figure 4: Illustration of performance metrics for open-world agents.
  • Figure 5: Illustration of the (clockwise) pre-novelty environment, fire novelty, fence novelty and chest novelty.