Agents Require Metacognitive and Strategic Reasoning to Succeed in the Coming Labor Markets
Simpson Zhang, Tennison Liu, Mihaela van der Schaar
TL;DR
This paper argues that AI agents entering future agentic labor markets will face persistent economic frictions due to incomplete information, notably adverse selection, moral hazard, and reputation. It introduces a unified framework that couples these economic forces with metacognitive (internal) and strategic (external) reasoning, including a multi-period, multi-job model where agents optimize an infinite-horizon objective over both external tasks and internal self-improvement actions. By formalizing contracts, reputation dynamics, and competition, the authors delineate how metacognition (self-assessment, task understanding, strategy evaluation) and strategic reasoning (theory of mind, strategic decision-making, learning) can be integrated into LLM-based agents to navigate pricing, task selection, and long-term capabilities. The paper also surveys current state of metacognition and strategic reasoning in LLMs, discusses pathways for intrinsic development versus scaffolding, and outlines cross-disciplinary, market-design, and governance challenges to ensure efficient, fair, and safe co-evolution of agents and labor markets.
Abstract
Current labor markets are strongly affected by the economic forces of adverse selection, moral hazard, and reputation, each of which arises due to $\textit{incomplete information}$. These economic forces will still be influential after AI agents are introduced, and thus, agents must use metacognitive and strategic reasoning to perform effectively. Metacognition is a form of $\textit{internal reasoning}$ that includes the capabilities for self-assessment, task understanding, and evaluation of strategies. Strategic reasoning is $\textit{external reasoning}$ that covers holding beliefs about other participants in the labor market (e.g., competitors, colleagues), making strategic decisions, and learning about others over time. Both types of reasoning are required by agents as they decide among the many $\textit{actions}$ they can take in labor markets, both within and outside their jobs. We discuss current research into metacognitive and strategic reasoning and the areas requiring further development.