Should I Have Expressed a Different Intent? Counterfactual Generation for LLM-Based Autonomous Control

TL;DR

This work addresses reliable counterfactual reasoning for LLM-based autonomous control in closed-loop agent–environment settings by formulating the problem as a structural causal model (SCM) and introducing counterfactual generation via abduction. The core contribution is conformal counterfactual generation (CCG), which uses test-time scaling and conformal language modeling to produce a reliable set of counterfactual reports $C_\lambda(T,X')$ with a formal guarantee that at least one candidate closely matches the true counterfactual $Y_{X'}(T)$ with probability $1-\epsilon$. The method jointly models the agent and environment, employing a neural posterior $q_\phi(U_Z|A,Z)$ for abduction and Gumbel–Max sampling for token-level decisions, and validates on a 5G network-control case where CG outperforms naive baselines in both KPI fidelity and semantic accuracy. The framework provides actionable reliability guarantees for operator questions like “what if I had expressed a different intent?” and has practical implications for debugging and validating prompt-driven autonomous systems in real-world network control contexts.

Abstract

Large language model (LLM)-powered agents can translate high-level user intents into plans and actions in an environment. Yet after observing an outcome, users may wonder: What if I had phrased my intent differently? We introduce a framework that enables such counterfactual reasoning in agentic LLM-driven control scenarios, while providing formal reliability guarantees. Our approach models the closed-loop interaction between a user, an LLM-based agent, and an environment as a structural causal model (SCM), and leverages test-time scaling to generate multiple candidate counterfactual outcomes via probabilistic abduction. Through an offline calibration phase, the proposed conformal counterfactual generation (CCG) yields sets of counterfactual outcomes that are guaranteed to contain the true counterfactual outcome with high probability. We showcase the performance of CCG on a wireless network control use case, demonstrating significant advantages compared to naive re-execution baselines.

Figures (8)

• Figure 1: (a) Example of the agentic framework studied in this work: A network operator provides an intent as a prompt to an LLM-based agent. The agent translates this prompt into an action, which is used to configure the network. The network executes the action, and returns environment feedback to the LLM in the form of KPIs, which the LLM processes and summarizes into a report for the operator. (b) LLM-environment interaction model: The user provides a prompt $X$, which the LLM maps to an action $A$. The environment executes action $A$ and produces feedback $Z$. The LLM then combines $(X, A, Z)$ to generate the response $Y$, which is returned to the user. (c) Given a factual episode $T = (X, A, Z, Y)$, the network operator chooses a counterfactual prompt $X'$. Given an observed factual episode $T = (X, A, Z, Y)$, our goal is to estimate the counterfactual report $Y_{X'}(T)$ that the user would have received, under the same conditions, had the input prompt been $X' \in \mathcal{E}(X)$ instead of $X$ (see Fig. \ref{['fig:example_prompt_1']} for an example).
• Figure 2: An example of a factual episode with a true and a generated counterfactual report. Emphasis (bold fonts) added to highlight the main differences between factual and counterfactual intents.
• Figure 3: SCM of the agent-environment pipeline: The user prompt $X$ influences the action $A$ chosen by the LLM, which in turn affects the environment feedback $Z$. Finally, the LLM combines $(X,A,Z)$ to generate the response $Y$. The exogenous variables $U_A$, $U_Z$, and $U_Y$ capture the randomness specific to the action, the environment, and the textual response, respectively, and are assumed independent of $X$.
• Figure 4: Given a factual episode $T=(X,A,Z,Y)$ and a counterfactual intent $X'$, conformal counterfactual generation (CCG) leverages test-time scaling to produce a set $C_\lambda(T,X')$ of counterfactual report candidates with guaranteed reliability with respect to the true counterfactual report.
• Figure 5: Average set loss as a function of the target set loss $\epsilon$ for CCG and fixed-budget baselines $k$-CG. The diagonal indicates ideal risk-controlled behavior.

Theorems & Definitions (2)

• Proposition 1
• proof