Plasticity as the Mirror of Empowerment
David Abel, Michael Bowling, André Barreto, Will Dabney, Shi Dong, Steven Hansen, Anna Harutyunyan, Khimya Khetarpal, Clare Lyle, Razvan Pascanu, Georgios Piliouras, Doina Precup, Jonathan Richens, Mark Rowland, Tom Schaul, Satinder Singh
TL;DR
This work develops a universal, agent-centric information-theoretic framework for measuring agency by introducing the generalized directed information (GDI), which extends directed information to arbitrary, overlapping time windows and bidirectional interactions. It defines two dual quantities, empowerment and plasticity, via GDI, and proves they are mirror images under opposite directions of influence, governed by a conservation law that partitions the information exchange between agent and environment. A tight trade-off is established: the sum of empowerment and plasticity is bounded by $m = \min\{(b-a+1) \log|\mathcal{O}|, (d-c+1) \log|\mathcal{A}|\}$, implying you cannot maximize both simultaneously for the same interval; in extreme cases, maximizing one forces the other to zero. The paper also provides necessary and sufficient conditions for nonzero plasticity, analyzes intuitive properties, and validates the framework with a simple Monte Carlo estimate of GDI in a Q-learning interacting with a two-armed bandit, illustrating the practical relevance for agent design and safety.
Abstract
Agents are minimally entities that are influenced by their past observations and act to influence future observations. This latter capacity is captured by empowerment, which has served as a vital framing concept across artificial intelligence and cognitive science. This former capacity, however, is equally foundational: In what ways, and to what extent, can an agent be influenced by what it observes? In this paper, we ground this concept in a universal agent-centric measure that we refer to as plasticity, and reveal a fundamental connection to empowerment. Following a set of desiderata on a suitable definition, we define plasticity using a new information-theoretic quantity we call the generalized directed information. We show that this new quantity strictly generalizes the directed information introduced by Massey (1990) while preserving all of its desirable properties. Under this definition, we find that plasticity is well thought of as the mirror of empowerment: The two concepts are defined using the same measure, with only the direction of influence reversed. Our main result establishes a tension between the plasticity and empowerment of an agent, suggesting that agent design needs to be mindful of both characteristics. We explore the implications of these findings, and suggest that plasticity, empowerment, and their relationship are essential to understanding agency