Reactive Environments for Active Inference Agents with RxEnvironments.jl

TL;DR

This paper introduces Reactive Environments, a comprehensive paradigm that facilitates complex multi-agent communication in nonequilibrium-Steady-State systems, and presents a Julia package RxEnvironments.jl, which is a specific implementation of Reactive Environments, where it utilizes a Reactive Programming style for efficient implementation.

Abstract

Active Inference is a framework that emphasizes the interaction between agents and their environment. While the framework has seen significant advancements in the development of agents, the environmental models are often borrowed from reinforcement learning problems, which may not fully capture the complexity of multi-agent interactions or allow complex, conditional communication. This paper introduces Reactive Environments, a comprehensive paradigm that facilitates complex multi-agent communication. In this paradigm, both agents and environments are defined as entities encapsulated by boundaries with interfaces. This setup facilitates a robust framework for communication in nonequilibrium-Steady-State systems, allowing for complex interactions and information exchange. We present a Julia package RxEnvironments.jl, which is a specific implementation of Reactive Environments, where we utilize a Reactive Programming style for efficient implementation. The flexibility of this paradigm is demonstrated through its application to several complex, multi-agent environments. These case studies highlight the potential of Reactive Environments in modeling sophisticated systems of interacting agents.

Figures (5)

• Figure 1: General communication protocol in an Active Inference environment containing two agents. The terms "Actuator" and "Sensor" are used from the agents' points of view. We see that both agents have a boundary with actuators and sensors with which they interact with the environment.
• Figure 2: Internal Entity logic is applied when an observation is received. On the left, we outline the steps an Entity should follow when processing an observation. On the right, we specify the a7RxEnvironments^^a7 functions that users can create to customize this behavior.
• Figure 3: Overview of interactions in the Mountain Car environment over time. The environment emits sensor information at a regular interval (2 Hz in this example), and whenever the agent emits an action, the environment instantaneously responds with proprioceptive feedback to the agent.
• Figure 4: Plot of the setup of our football environment, showing the pitch and the 22 players. The ball is positioned on the center spot. An animation of this environment where we send random run commands to players can be found https://youtu.be/24ZSPcVDOqc.
• Figure 5: Schematic of the subscriptions in the hearing aid environment.

Theorems & Definitions (2)

• definition thmcounterdefinition
• definition thmcounterdefinition