Test-Time Compute Games

TL;DR

The paper tackles the misalignment between profit-driven test-time compute (TTC) in LLM-as-a-service and social welfare. It formalizes TTC markets as a normal-form game, analyzes equilibria through a generalized ordinal potential framework, and proves that standard TTC markets are socially inefficient (positive price of anarchy). To address this, it proposes a reverse second-price auction in which a third-party platform incentivizes providers to choose TTC levels that maximize social welfare, yielding a welfare-optimal outcome with dominant strategies. Empirical results across GSM8K, GPQA, and AIME with Llama, Qwen, and DeepSeek-R1 models show meaningful welfare gains under the auction, particularly for non-reasoning settings, while still highlighting limitations and avenues for real-world deployment and extensions. Overall, the work provides both theoretical guarantees and practical insights for welfare-centered market designs in TTC-enabled LLM services, with potential for broader applicability in AI-driven infrastructure markets.

Abstract

Test-time compute has emerged as a promising strategy to enhance the reasoning abilities of large language models (LLMs). However, this strategy has in turn increased how much users pay cloud-based providers offering LLM-as-a-service, since providers charge users for the amount of test-time compute they use to generate an output. In our work, we show that the market of LLM-as-a-service is socially inefficient: providers have a financial incentive to increase the amount of test-time compute, even if this increase contributes little to the quality of the outputs. To address this inefficiency, we introduce a reverse second-price auction mechanism where providers bid their offered price and (expected) quality for the opportunity to serve a user, and users pay proportionally to the marginal value generated by the winning provider relative to the second-highest bidder. To illustrate and complement our theoretical results, we conduct experiments with multiple instruct models from the $\texttt{Llama}$ and $\texttt{Qwen}$ families, as well as reasoning models distilled from $\texttt{DeepSeek-R1}$, on math and science benchmark datasets.

Figures (38)

• Figure 1: User values offered by providers in a test-time compute game. The figure shows the user values $V_i(\theta)$ offered by providers in a test-time compute game $\mathcal{G}$, as a function of their test-time compute $\theta$. Panel (a) corresponds to a test-time compute game with $N=9$ providers serving non-reasoning models from the Llama and Qwen families, where providers use best-of-n across $\theta$ samples as their test-time compute method. Panel (b) corresponds to a game with $N=3$ providers serving reasoning models distilled from DeepSeek-R1, where $\theta$ represents reasoning effort, defined by binning the model outputs into quantiles based on the number of reasoning tokens (see Appendix \ref{['app:experimental-details']}). In both games, providers serve queries $Q$ from the GSM8K dataset, and we consider that each (average) percentage point of accuracy offers a value of $\$0.008$ to the users.
• Figure 2: Dynamics of a test-time compute game. The figure illustrates the better-response dynamics of a test-time compute game $\mathcal{G}$ when providers sequentially select a test-time compute level that increases their utility. The upper panels show the compute levels $\theta^t$ selected by each provider and the resulting market inefficiency (dashed black curve), defined as $\max_\boldsymbol{\theta} \text{W}(\boldsymbol{\theta})/\text{W}(\boldsymbol{\theta}^t)$, and the lower panels show the market share of each provider. Panel (a) and Panel (b) correspond to the games described in Figure \ref{['fig:main-values']}, where providers serve, respectively, non-reasoning from the Llama and Qwen families, and reasoning models distilled from DeepSeek-R1. Here, we set $\beta=1000$.
• Figure 3: Accuracy of Llama and Qwen models on GSM8K using majority voting and best-of-n. Panels (a) and (b) show the average accuracy of various LLMs from the Llama and Qwen families over questions from the GSM8K dataset, where the responses of the models are obtained using majority voting or best-of-n, respectively. Panel (c) shows, as a function of the number of samples used to generate the response, the total number of tokens that the models generate to obtain the response to each question, averaged across questions. We show $95 \%$ confidence intervals obtained by bootstrapping $50$ times.
• Figure 4: Accuracy of reasoning models distilled from DeepSeek-R1 on GSM8K using chain-of-thought. Panel (a) shows the average accuracy of various reasoning models from the Llama and Qwen families distilled from DeepSeek-R1 over questions from the GSM8K dataset. Here, the reasoning effort is defined by binning the model outputs into quantiles based on the number of reasoning tokens (see Appendix \ref{['app:experimental-details']}). Panel (c) shows, as a function of the reasoning effort, the total number of tokens (including reasoning and non-reasoning tokens) that the models generate as a response to each question, averaged across questions. We show $95 \%$ confidence intervals obtained by bootstrapping $50$ times. Refer to Appendix \ref{['app:experimental-details']} for further details regarding the evaluation of the models.
• Figure 5: Accuracy of Llama and Qwen models on AIME using majority voting and best-of-n. Panels (a) and (b) show the average accuracy of various LLMs from the Llama and Qwen families over questions from the AIME dataset, where the responses of the models are obtained using majority voting or best-of-n, respectively. Panel (c) shows, as a function of the number of samples used to generate the response, the total number of tokens that the models generate to obtain the response to each question, averaged across questions. Here, we compute the accuracies for majority voting, and best-of-n are computed across the same outputs, and hence both majority voting and best-of-n generate the exact same number of average tokens. We show $95 \%$ confidence intervals obtained by bootstrapping $50$ times. Refer to Appendix \ref{['app:experimental-details']} for further details regarding the evaluation of the models.

Theorems & Definitions (9)

• Theorem 1
• Theorem 2
• Theorem 3
• Theorem 4
• Theorem 5
• Lemma 1
• proof
• Lemma 2
• proof