Modeling Sustainable Resource Management using Active Inference

TL;DR

Problem: How can an agent maintain its well-being while ensuring sustainable resource use when consumption alters future availability? Approach: an active-inference agent with a generative model over hidden states $s_1$ (food left) and $s_2$ (satiety), observations $o$, action set {eat, don't eat}, and parameters $A,B,C,D$, analyzed in static and dynamic environments with a planning horizon of up to $3$ steps. Key contributions: (i) demonstration that long-horizon planning and learning of transition dynamics enable sustainable behavior; (ii) analysis of the roles of priors (through $C$) and model accuracy; (iii) identification of inertia and learning benefits in dynamic environments. Significance: provides a principled framework for understanding, predicting, and shaping sustainable behavior in adaptive agents and informs extensions to multi-resource and multi-agent systems.

Abstract

Active inference helps us simulate adaptive behavior and decision-making in biological and artificial agents. Building on our previous work exploring the relationship between active inference, well-being, resilience, and sustainability, we present a computational model of an agent learning sustainable resource management strategies in both static and dynamic environments. The agent's behavior emerges from optimizing its own well-being, represented by prior preferences, subject to beliefs about environmental dynamics. In a static environment, the agent learns to consistently consume resources to satisfy its needs. In a dynamic environment where resources deplete and replenish based on the agent's actions, the agent adapts its behavior to balance immediate needs with long-term resource availability. This demonstrates how active inference can give rise to sustainable and resilient behaviors in the face of changing environmental conditions. We discuss the implications of our model, its limitations, and suggest future directions for integrating more complex agent-environment interactions. Our work highlights active inference's potential for understanding and shaping sustainable behaviors.

Figures (11)

• Figure 1: The agent’s generative model encodes beliefs about the causal structure of the environment and how its actions affect the state of the world. The true state of the environment is represented by two hidden state factors - the availability of food (s1) and the agent’s satiety (s2). The prior preference C matrix specifies the agent’s innate drives or goals, in this case a strong preference for being satiated. The starting conditions are specified by the initial state distribution, D. Here, food is initially present but the agent is not satiated. The agent has two observation modalities - the presence of food (o1) and its own satiety level (o2). The agent can select between two actions at each time step - "eat" or "do not eat". We have two hidden state factors: food left and satiety. For the "do not eat" action, for Case 1 the B matrix is an identity matrix, as this action does not change the state, while for Case 2 it changes, since not eating leads to an increase in available food. When the agent chooses the "eat" action, if food is present, the states will transition by reducing "food left" by 1 (down to a minimum of 0) state and by increasing "satiety" by 1 (to a maximum of 2).
• Figure 2: Case 2, showing the survival time of agents with both learning and no learning when the agent starts with a random B matrix. The survival time plot for each run averaged over 10 agents, shows that the agents can quickly learn to survive by acting in a sustainable way.
• Figure 3: Case 1. The three plots above show the expected behavior. At time step 0, food is available (food left = 0), and the satiety level is low (satiety = 0). Due to the agent's strong preference for high level of satiety, it keeps eating at subsequent time steps and the satiety increases. Since the environment is static, the food is always present.
• Figure 4: Case 1.1 - where the agent is given incorrect A and B matrices, introducing errors in its perception and beliefs about state transitions. The top plot shows the agent's actions over time. The pattern is more erratic compared to the standard Case 1, as the agent is confused. The middle plot shows the food left observations. Food availability remains constant at 1 throughout the simulation since the environment is static. The bottom plot shows the agent's satiety over time. Satiety level fluctuates more. This indicates that the agent's ability to maintain a stable, high satiety state is impaired by the incorrect perception and planning models.
• Figure 5: Prior preference for Case 2. The agent's preferences are changed so that, unlike case 1 and 1.1 it no longer has a preference over food left. Its only non-uniform preference is to have a preference over satiety.