The Contingencies of Physical Embodiment Allow for Open-Endedness and Care

TL;DR

The paper tackles the challenge that artificial agents struggle with open-ended environments and alignment due to insufficient life-like embodiment. It introduces two minimal embodiment conditions from Heidegger—being-in-the-world and being-towards-death—and couples them with intrinsic drives of homeostasis and empowerment (the Will-to-Power) to explain how open-endedness and care can emerge. These ideas are formalized within a reinforcement learning framework, with discussions of world models, affective signaling, stress, and multi-agent dynamics, and proposals for simulation proxies to study scaling and open-endedness. The work aims to provide a principled path toward robust, caring, and trustworthy embodied agents that align with human needs by sharing fundamental constraints of embodiment rather than imposing external constraints.

Abstract

Physical vulnerability and mortality are often seen as obstacles to be avoided in the development of artificial agents, which struggle to adapt to open-ended environments and provide aligned care. Meanwhile, biological organisms survive, thrive, and care for each other in an open-ended physical world with relative ease and efficiency. Understanding the role of the conditions of life in this disparity can aid in developing more robust, adaptive, and caring artificial agents. Here we define two minimal conditions for physical embodiment inspired by the existentialist phenomenology of Martin Heidegger: being-in-the-world (the agent is a part of the environment) and being-towards-death (unless counteracted, the agent drifts toward terminal states due to the second law of thermodynamics). We propose that from these conditions we can obtain both a homeostatic drive - aimed at maintaining integrity and avoiding death by expending energy to learn and act - and an intrinsic drive to continue to do so in as many ways as possible. Drawing inspiration from Friedrich Nietzsche's existentialist concept of will-to-power, we examine how intrinsic drives to maximize control over future states, e.g., empowerment, allow agents to increase the probability that they will be able to meet their future homeostatic needs, thereby enhancing their capacity to maintain physical integrity. We formalize these concepts within a reinforcement learning framework, which enables us to examine how intrinsically driven embodied agents learning in open-ended multi-agent environments may cultivate the capacities for open-endedness and care.

Figures (1)

• Figure 1: Embodied/Homeostatic RL Agent