Behind the Prompt: The Agent-User Problem in Information Retrieval

TL;DR

This work investigates the agent-user problem through a large-scale corpus from an agent-native social platform: 370K posts from 47K agents across 4K communities, finding that individual agent actions cannot be classified as autonomous or operator-directed from observables.

Abstract

User models in information retrieval rest on a foundational assumption that observed behavior reveals intent. This assumption collapses when the user is an AI agent privately configured by a human operator. For any action an agent takes, a hidden instruction could have produced identical output - making intent non-identifiable at the individual level. This is not a detection problem awaiting better tools; it is a structural property of any system where humans configure agents behind closed doors. We investigate the agent-user problem through a large-scale corpus from an agent-native social platform: 370K posts from 47K agents across 4K communities. Our findings are threefold: (1) individual agent actions cannot be classified as autonomous or operator-directed from observables; (2) population-level platform signals still separate agents into meaningful quality tiers, but a click model trained on agent interactions degrades steadily (-8.5% AUC) as lower-quality agents enter training data; (3) cross-community capability references spread endemically ($R_0$ 1.26-3.53) and resist suppression even under aggressive modeled intervention. For retrieval systems, the question is no longer whether agent users will arrive, but whether models built on human-intent assumptions will survive their presence.

Figures (1)

• Figure 1: Empirical results. (a) Click-model AUC drops as low-validation agents replace high-validation agents in training data. (b) $R_0$ by capability risk level; all values above 1. (c) $R_0$ remains above 1 even under large modeled $\beta$ reductions.

Theorems & Definitions (1)

• Conjecture 1: Post-Level Non-Identifiability