Alignment, Agency and Autonomy in Frontier AI: A Systems Engineering Perspective
Krti Tallam
TL;DR
This paper examines how alignment, agency, and autonomy are defined and used in frontier AI, arguing disciplinary variations complicate design and regulation. It analyzes historical roots and current challenges, including specification gaming, goal misgeneralization, and instrumental convergence, and uses case studies (Tesla Autopilot, MCAS, CICERO, AlphaZero, AutoGPT) to illustrate governance risks. The authors propose a taxonomy of AI agency levels and advocate for robust governance frameworks, corrigibility, and ongoing human oversight. The work contributes a structured framework for measuring and governing AI along the three axes, concrete lessons from real-world systems, and open research questions for safe deployment of agentic AI.
Abstract
As artificial intelligence scales, the concepts of alignment, agency, and autonomy have become central to AI safety, governance, and control. However, even in human contexts, these terms lack universal definitions, varying across disciplines such as philosophy, psychology, law, computer science, mathematics, and political science. This inconsistency complicates their application to AI, where differing interpretations lead to conflicting approaches in system design and regulation. This paper traces the historical, philosophical, and technical evolution of these concepts, emphasizing how their definitions influence AI development, deployment, and oversight. We argue that the urgency surrounding AI alignment and autonomy stems not only from technical advancements but also from the increasing deployment of AI in high-stakes decision making. Using Agentic AI as a case study, we examine the emergent properties of machine agency and autonomy, highlighting the risks of misalignment in real-world systems. Through an analysis of automation failures (Tesla Autopilot, Boeing 737 MAX), multi-agent coordination (Metas CICERO), and evolving AI architectures (DeepMinds AlphaZero, OpenAIs AutoGPT), we assess the governance and safety challenges posed by frontier AI.