GTAlign: Game-Theoretic Alignment of LLM Assistants for Social Welfare

TL;DR

GTAlign reframes user–LLM interactions as strategic games and injects game-theoretic reasoning into both reasoning and training. It introduces a four-block reasoning chain that computes payoff matrices and maximizes social welfare, with a Cobb-Douglas aggregation W(U,L)=√(U L) to balance user satisfaction and model efficiency. The framework also enables inference-time steering by modifying the payoff structure to adapt to pricing policies, allowing welfare-aware behavior without retraining. Across diverse tasks and OOD benchmarks, GTAlign improves reasoning efficiency, answer quality, and social welfare, with human studies showing higher user satisfaction and strong alignment with welfare gains. The work highlights game theory as a principled lens for LLM alignment, offering scalable, interpretable mechanisms for cooperative and welfare-oriented dialogue systems.

Abstract

Large Language Models (LLMs) have achieved remarkable progress in reasoning, yet sometimes produce responses that are suboptimal for users in tasks such as writing, information seeking, or providing practical guidance. Conventional alignment practices typically assume that maximizing model reward also maximizes user welfare, but this assumption frequently fails in practice: models may over-clarify or generate overly verbose reasoning when users prefer concise answers. Such behaviors resemble the prisoner's dilemma, where individually rational choices lead to socially suboptimal outcomes. The fundamental challenge is the lack of a principled decision making mechanism that mutually benefits both the LLM and the user. We propose Game-Theoretic Alignment (GTAlign), an alignment framework that integrates game-theoretic decision making into both reasoning and training. During reasoning, the model explicitly treats user-LLM interaction as a strategic game: it constructs payoff matrices within its reasoning chain to estimate welfare for both itself and the user, and then selects actions that are mutually beneficial. During training, we introduce a social welfare reward that reinforces cooperative responses, aligning model behavior with socially efficient outcomes. In addition, we introduce an inference technique that leverages game-theoretic reasoning to dynamically adapt LLM's response when pricing policies of LLM service change. Extensive experiments demonstrate that GTAlign substantially improves reasoning efficiency, answer quality, and social welfare compared to baselines across diverse tasks. The code is available at https://github.com/ulab-uiuc/GTAlign .

Figures (6)

• Figure 1: Game theory could optimize user-LLM interaction. Default Interaction settles into the Prisoner's Dilemma. Preferably, LLM would guide the conversation to the jointly optimal outcome.
• Figure 2: GTAlign generates responses using game-theoretic reasoning. A social welfare reward, calculated from the final response, is used for reinforcement learning.
• Figure 3: Directly training LLMs with RL on classic games shows high variance and no convergence. We train Qwen2.5-7B-Instruct on Sequential Prisoner's Dilemma, Rubinstein Bargaining, and Sequential Signaling games with RL, showing the reward over training steps.
• Figure 4: GTAlign can steer LLM behavior during inference. When the LLM pricing policy switches, we can steer LLM behavior by modifying the payoff matrix.
• Figure 5: Illustration of Pareto efficiency.