Fully Autonomous AI Agents Should Not be Developed
Margaret Mitchell, Avijit Ghosh, Alexandra Sasha Luccioni, Giada Pistilli
TL;DR
The paper argues that fully autonomous AI agents should not be developed due to safety, privacy, and trust risks that escalate with autonomy. It formalizes a definition and a leveled framework for AI agents, then analyzes how increasing autonomy interacts with a comprehensive value taxonomy (accuracy, assistiveness, efficiency, equity, flexibility, humanlikeness, privacy, safety, security, trust). Through methodology grounded in applied ethics and a reliance on base models like LLMs, the authors show that while some benefits arise, the potential for cascading harms and loss of human oversight grows with each higher level of autonomy. The work advocates adopting a spectrum of agent autonomy, implementing robust human-override and safety mechanisms, and prioritizing semi-autonomous systems to harness benefits while mitigating catastrophic risks. It emphasizes governance, oversight, and safety guarantees as essential to responsible AI development going forward.
Abstract
This paper argues that fully autonomous AI agents should not be developed. In support of this position, we build from prior scientific literature and current product marketing to delineate different AI agent levels and detail the ethical values at play in each, documenting trade-offs in potential benefits and risks. Our analysis reveals that risks to people increase with the autonomy of a system: The more control a user cedes to an AI agent, the more risks to people arise. Particularly concerning are safety risks, which affect human life and impact further values.