Accounting for AI and Users Shaping One Another: The Role of Mathematical Models

TL;DR

The paper addresses how AI systems and users mutually shape each other, arguing that formal interaction models—mathematical, coupled-dynamical systems—provide a principled way to specify, monitor, anticipate, and control these dynamics. By framing AI-user interactions as coupled state updates $x_t$ and outputs $y_t$ with $x_{t+1}=f(x_t,u_t,w_t)$ and $y_t=g(x_t,u_t,w_t)$, it unifies various domains (notably content recommender systems) under a common language and highlights four use-cases: specification, monitoring, counterfactual analysis, and intervention design. The authors survey the nascent literature in content recommendation, showing how viewer and creator dynamics have been modeled (often with distinct styles such as nonlinear dynamics for viewers and game theory for creators) and identify gaps—particularly in specifying and monitoring interactions and in cross-type integration. They propose design axes (style, granularity, mathematical complexity, measurability) to guide model construction and emphasize opportunities to broaden the scope to multiple interaction types and domains beyond recommender systems. Overall, formal interaction models hold promise to improve AI system design and governance by making the bidirectional effects between AI and users tractable, verifiable, and controllable, with implications for policy, UI design, and engineering practice across AI-enabled ecosystems.

Abstract

As AI systems enter into a growing number of societal domains, these systems increasingly shape and are shaped by user preferences, opinions, and behaviors. However, the design of AI systems rarely accounts for how AI and users shape one another. In this position paper, we argue for the development of formal interaction models which mathematically specify how AI and users shape one another. Formal interaction models can be leveraged to (1) specify interactions for implementation, (2) monitor interactions through empirical analysis, (3) anticipate societal impacts via counterfactual analysis, and (4) control societal impacts via interventions. The design space of formal interaction models is vast, and model design requires careful consideration of factors such as style, granularity, mathematical complexity, and measurability. Using content recommender systems as a case study, we critically examine the nascent literature of formal interaction models with respect to these use-cases and design axes. More broadly, we call for the community to leverage formal interaction models when designing, evaluating, or auditing any AI system which interacts with users.

Figures (2)

• Figure 1: Left: illustration of a dynamical system where a model interacts with an unspecified environment. Right: illustration of a formal interaction model as a coupled dynamical system between an AI system and users. The variable $x$ captures the internal state, the function $f$ captures the state update, the function $g$ captures the measurement equation, the variable $u$ denotes inputs, the variable $y$ denotes outputs (and inputs, in the case of the coupled dynamical system), and the variable $w$ captures noise or other disturbance (Section \ref{['subsec:overview']}).
• Figure 2: Illustration of a formal interaction model for a content recommendation system as a coupled dynamical system between the recommender system $(x^r, y^r, f^r, g^r)$, viewers $(x^v, y^v, f^v, g^v)$, and creators $(x^c, y^c, f^c, g^c)$. The variable $x$ captures the internal state, the function $f$ captures the state update, the function $g$ captures the measurement equation, and the variable $y$ denotes outputs and inputs (Section \ref{['subsec:overview']} and Section \ref{['sec:casestudy']}).

Theorems & Definitions (1)

• Example 1