Decision Aggregation under Quantal Response

Abstract

The effectiveness of collective decision-making is often challenged by the bounded rationality and inherent stochasticity of individual agents. We investigate this by analyzing how to aggregate decisions from n experts, each receiving a private signal about an unknown state. Assuming signals are conditionally independent and identically distributed, we depart from the fully rational paradigm and model expert behavior using quantal response, a stochastic choice model capturing bounded rationality. Within a minimax regret framework, we show that majority voting is the optimal robust aggregator when individual rationality falls below a certain threshold. Interestingly, such groups can outperform perfectly rational agents, as their decision randomness encodes weak but informative signals lost in deterministic behavior. We validate these findings using large language models (LLMs), which naturally exhibit quantal response via their temperature parameter. Aggregating moderately stochastic LLM outputs significantly improves accuracy on complex reasoning tasks, highlighting bounded rationality not as a limitation, but as a potential strength in collective intelligence.

Figures (6)

• Figure 1: Comparison of rational (Mia) vs. bounded-rational (John) decision-making through unified quantal response functions.
• Figure 2: Logistic Regression Results Across Temperatures. The plots illustrate the relationship between posterior probability (x-axis) and decision proportion (y-axis) at three temperature settings ($t = 0.0, 0.5, 1.0$). Each gray dot represents a specific scenario, showing the decision proportion of GPT-4o-mini for that scenario. The fitted quantal response curve (orange line) represents the model's predicted behavior, with the fitted coefficient $\lambda$ quantifying the rationality of decision-making. Specifically, $\lambda \to \infty$ at $t = 0.0$, $\lambda = 13.25$ at $t = 0.5$, and $\lambda = 8.93$ at $t = 1.0$. The results demonstrate increased randomness as temperature rises, aligning with the quantal response model's predictions.
• Figure 3: Performance of Majority/Plurality Voting Aggregation. Panel (a) presents the expected utility of the majority-vote aggregator ($f^{maj}$) in the Bayesian decision-making task, where different temperature settings ($t = 0.0, 0.5, 1.0$) and numbers of experts ($n = 1, 3, 5$) are evaluated. Panel (b) shows the accuracy of the plurality-vote aggregator ($f^{plu}$) when applied to the multiple-choice question answering task using the MathQA dataset. In both panels, error bars represent the standard error of the mean (SEM), which quantifies the uncertainty in the estimated mean. Both studies show the same pattern: increasing the number of experts ($n$) improves aggregation performance. When $n = 1$, higher temperature ($t$) decreases performance due to increased randomness. When $n \geq 3$, higher temperature improves performance as diversity enhances aggregation effectiveness. These results highlight that while randomness degrades individual decisions, it benefits collective decision-making when properly aggregated, which aligns with our theoretical findings.
• Figure 4: Threshold $g(n)$ vs. Group Size $n$ ($n = 3$ to $20$) The plot begins at $n = 3$ because when $n \leq 2$, the majority voting is always the optimal robust aggregator, independent of $\lambda$. For $n \in [3,20]$, $g(n)$ decreases with increasing $n$, with every even $n$ satisfying $g(n) = g(n-1)$. These properties indicate that as group size grows, the bounded rationality threshold becomes stricter, representing a lower rationality level to preserve the optimality of the majority voting rule.
• Figure 5: Regret Comparison: Majority Voting vs. Optimal Robust Aggregator (Varying Rationality and Group Size). These plots illustrate the relationship between the rationality level $\lambda$ (on the x-axis) and the regret (on the y-axis) for three different group sizes: $n = 1, 3,$ and $5$. Regret is the maximum regret over all report structures induced by all c.i.i.d. signal structures. The solid curves represent the regret incurred by the majority voting, $f^{maj}$, and the optimal robust aggregator, $\text{opt}_{\hat{\Theta}^{ciid}}$. The dotted line indicates the threshold function $g(n)$. When the performance of majority voting matches that of the optimal robust aggregator (i.e., their regrets are equal), majority voting is optimal. With a single expert, any aggregation rule that follows that expert's decision, including majority voting, is trivially optimal. As $\lambda$ increases, reflecting increased rationality, the regret decreases. When $n \geq 3$ and $\lambda \leq g(n)$, majority voting remains optimal. However, the threshold $g(n)$ is not tight; majority voting may still be optimal even when $\lambda$ exceeds $g(n)$. Nevertheless, for $n=5$ and sufficiently large $\lambda$, majority voting is no longer optimal. Furthermore, as $\lambda$ increases, the regret of both majority voting and the optimal robust aggregator initially decreases and then increases. This suggests that a moderate degree of bounded rationality can, in fact, improve aggregation performance.

Theorems & Definitions (31)

• Theorem 3.1: Main Theorem
• Lemma A.2: Dimension Reduction
• proof : Proof of Lemma \ref{['lem:dimension_reduction']}
• Lemma A.3
• proof : Proof of Lemma \ref{['lem:sig2rep']}
• Lemma A.4
• proof : Proof of Lemma \ref{['lem:no4cop']}
• Lemma A.5
• proof : Proof of Lemma \ref{['lem:4point_cop']}
• Lemma A.6