More Agents Helps but Adversarial Robustness Gap Persists

TL;DR

This study evaluates the robustness of multi-agent LLM collaboration for mathematics question answering under adversarial input perturbations. Using Agent Forest, a simple sampling-and-voting framework, the authors assess six open-source models across four benchmarks and seven agent counts, under both synthetic punctuation noise and human-like typos (WikiTypo, R2ATA). Key findings show that while increasing the number of collaborating agents reliably boosts accuracy—with the largest gains from $n=1$ to $n=5$ and diminishing returns beyond $\approx 10$—the adversarial robustness gap remains substantial and largely uninfluenced by larger ensembles. Noise type matters most: human-like typos (WikiTypo) impose the largest accuracy gaps and ASR, punctuation perturbations are mitigated with modest collaboration, and R2ATA effects lie between. The results highlight that multi-agent collaboration improves average reasoning performance but does not eliminate fragility to realistic input perturbations, guiding future work toward noise-aware sampling, verifier-assisted agents, and typos-focused augmentation. The findings have practical implications for deploying ensemble-based reasoning systems in imperfect real-world text inputs.

Abstract

When LLM agents work together, they seem to be more powerful than a single LLM in mathematical question answering. However, are they also more robust to adversarial inputs? We investigate this question using adversarially perturbed math questions. These perturbations include punctuation noise with three intensities (10, 30, and 50 percent), plus real-world and human-like typos (WikiTypo, R2ATA). Using a unified sampling-and-voting framework (Agent Forest), we evaluate six open-source models (Qwen3-4B/14B, Llama3.1-8B, Mistral-7B, Gemma3-4B/12B) across four benchmarks (GSM8K, MATH, MMLU-Math, MultiArith), with various numbers of agents n from one to 25 (1, 2, 5, 10, 15, 20, 25). Our findings show that (1) Noise type matters: punctuation noise harm scales with its severity, and the human typos remain the dominant bottleneck, yielding the largest gaps to Clean accuracy and the highest ASR even with a large number of agents. And (2) Collaboration reliably improves accuracy as the number of agents, n, increases, with the largest gains from one to five agents and diminishing returns beyond 10 agents. However, the adversarial robustness gap persists regardless of the agent count.

Figures (7)

• Figure 1: Average accuracy across datasets and models as a function of agent count. Lines denote noise types (Clean, Punct-$10$/$30$/$50$, WikiTypo, R2ATA). Markers show mean accuracy across datasets.
• Figure 2: Overview of the experimental framework for evaluating multi-LLM-agent robustness. The system processes input questions from four datasets under Clean and noisy conditions. For each condition, a single LLM generates multiple independent answers (LLM$_1$ to LLM$_n$), which are then aggregated by majority voting to produce the final answer. The performance of the single-agent and multi-agent systems is compared against the ground truth in terms of accuracy and attack success rate.
• Figure 3: Accuracy averaged across datasets by noise type (Clean, Punct-$10$/$30$/$50$, WikiTypo, R2ATA) as a function of agent count. Larger models (Gemma3-12B and Qwen3-14B) maintain higher absolute accuracy across conditions, while smaller (and older) models (Llama3.1 and Mistral-7B) show steeper relative gains with increasing agents. However, new small models (Gemma3-4B and Qwen3-4B) show similar performances to their larger counterparts.
• Figure 4: Attack Success Rate averaged across datasets, shown by noise category (Punctuation, WikiTypo, R2ATA). Bars represent different models at varying agent counts. Punctuation noise vulnerability is most effectively reduced with more agents. WikiTypo remains the most challenging across all models, and R2ATA lies in between, initially harmful for single agents but increasingly mitigated through collaboration.
• Figure 5: Accuracy averaged across models, shown separately for each dataset as a function of agent count. Collaboration improves accuracy under all noise types, with the largest gains on MATH, MultiArith, and GSM8K. In contrast, MMLU shows smaller absolute drops under noise and saturates more quickly with increasing agent count.