Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Agency Is Frame-Dependent

David Abel, André Barreto, Michael Bowling, Will Dabney, Shi Dong, Steven Hansen, Anna Harutyunyan, Khimya Khetarpal, Clare Lyle, Razvan Pascanu, Georgios Piliouras, Doina Precup, Jonathan Richens, Mark Rowland, Tom Schaul, Satinder Singh

TL;DR

The paper argues that agency is not an intrinsic, frame-invariant property of a system but is frame-dependent within reinforcement learning. It introduces a four-part canonical account of agency—boundary, source of action, normativity, and adaptivity—and defines an agent reference frame as a collection of commitments that determine how these properties are assessed. By showing each property is contingent on arbitrary frame choices (e.g., boundaries, causal variables, and criteria for goal-pursuit and adaptation), the authors argue that attribution of agency requires explicit frame specification. The work outlines implications for understanding intelligence and agency together, and calls for formalizing agent reference frames and exploring principled frame-selection criteria, linking the perspective to Marr's levels and Dennett's stances. Overall, it advocates a frame-based science of agency with potential to deepen the theoretical foundations of RL and related disciplines.

Abstract

Agency is a system's capacity to steer outcomes toward a goal, and is a central topic of study across biology, philosophy, cognitive science, and artificial intelligence. Determining if a system exhibits agency is a notoriously difficult question: Dennett (1989), for instance, highlights the puzzle of determining which principles can decide whether a rock, a thermostat, or a robot each possess agency. We here address this puzzle from the viewpoint of reinforcement learning by arguing that agency is fundamentally frame-dependent: Any measurement of a system's agency must be made relative to a reference frame. We support this claim by presenting a philosophical argument that each of the essential properties of agency proposed by Barandiaran et al. (2009) and Moreno (2018) are themselves frame-dependent. We conclude that any basic science of agency requires frame-dependence, and discuss the implications of this claim for reinforcement learning.

Agency Is Frame-Dependent

TL;DR

The paper argues that agency is not an intrinsic, frame-invariant property of a system but is frame-dependent within reinforcement learning. It introduces a four-part canonical account of agency—boundary, source of action, normativity, and adaptivity—and defines an agent reference frame as a collection of commitments that determine how these properties are assessed. By showing each property is contingent on arbitrary frame choices (e.g., boundaries, causal variables, and criteria for goal-pursuit and adaptation), the authors argue that attribution of agency requires explicit frame specification. The work outlines implications for understanding intelligence and agency together, and calls for formalizing agent reference frames and exploring principled frame-selection criteria, linking the perspective to Marr's levels and Dennett's stances. Overall, it advocates a frame-based science of agency with potential to deepen the theoretical foundations of RL and related disciplines.

Abstract

Agency is a system's capacity to steer outcomes toward a goal, and is a central topic of study across biology, philosophy, cognitive science, and artificial intelligence. Determining if a system exhibits agency is a notoriously difficult question: Dennett (1989), for instance, highlights the puzzle of determining which principles can decide whether a rock, a thermostat, or a robot each possess agency. We here address this puzzle from the viewpoint of reinforcement learning by arguing that agency is fundamentally frame-dependent: Any measurement of a system's agency must be made relative to a reference frame. We support this claim by presenting a philosophical argument that each of the essential properties of agency proposed by Barandiaran et al. (2009) and Moreno (2018) are themselves frame-dependent. We conclude that any basic science of agency requires frame-dependence, and discuss the implications of this claim for reinforcement learning.

Paper Structure

This paper contains 15 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Agency.
  3. The Puzzle of Agency.
  4. Agency is Frame-Dependent
  5. Main Claim.
  6. What is a Reference Frame?
  7. (1) Individuality.
  8. (2) Source of Action.
  9. (3) Normativity.
  10. (4) Adaptivity.
  11. Discussion
  12. Intelligence and Agency.
  13. Reference Frames.
  14. Choosing a Reference Frame.
  15. Marr's Levels, Dennett's Stances.

Figures (1)

  • Figure 1: (Left) A four-part account of agency due largely to barandiaran2009defining: a system such as a thermostat has agency if it has (1) a boundary, (2) is the source of its own actions, (3) has a goal, and (4) adaptively selects outputs based on inputs to pursue this goal. (Right) Our main claim: A determination of the agency of a system, such as a thermostat, is dependent on a choice of reference frame. The two reference frames depicted make different commitments about how we measure the four essential conditions of agency. For example, we could draw the boundary around our thermostat in several different ways, or understand the goal of the thermostat in different ways.

Theorems & Definitions (4)

  • Claim 1: Adapted from jiang2019value, jiang2019value and harutyunyan2020what, harutyunyan2020what
  • Claim 2: Adapted from kenton2023discovering, kenton2023discovering
  • Claim 3
  • Claim 4: Adapted from zadeh1963definition, zadeh1963definition and Theorem 3.1 of abel2023crl, abel2023crl