Robustness as an Emergent Property of Task Performance

TL;DR

The paper investigates whether robustness, defined as output consistency across prompt-like variations, emerges as models saturate on a given task. It demonstrates a strong positive correlation between task performance and robustness across six datasets and 24 configuration variants, quantified by a slope of $slope = 1.05$ with $R^2 = 0.924$, indicating robustness increases as benchmarks approach saturation. The authors argue robustness is primarily driven by task-specific competence rather than intrinsic model properties, suggesting robustness will co-emerge with performance on new tasks. Practically, this implies robustness may be less of a separate objective for researchers and more a natural indicator of deployment readiness for practitioners, though the study is limited to classification tasks and open-model evaluations.

Abstract

Robustness is often regarded as a critical future challenge for real-world applications, where stability is essential. However, as models often learn tasks in a similar order, we hypothesize that easier tasks will be easier regardless of how they are presented to the model. Indeed, in this paper, we show that as models approach high performance on a task, robustness is effectively achieved. Through an empirical analysis of multiple models across diverse datasets and configurations (e.g., paraphrases, different temperatures), we find a strong positive correlation. Moreover, we find that robustness is primarily driven by task-specific competence rather than inherent model-level properties, challenging current approaches that treat robustness as an independent capability. Thus, from a high-level perspective, we may expect that as new tasks saturate, model robustness on these tasks will emerge accordingly. For researchers, this implies that explicit efforts to measure and improve robustness may warrant reduced emphasis, as such robustness is likely to develop alongside performance gains. For practitioners, it acts as a sign that indeed the tasks that the literature deals with are unreliable, but on easier past tasks, the models are reliable and ready for real-world deployment.

Figures (12)

• Figure 1: Linear regression of robustness on performance. Robustness increases slightly faster than performance, with a slope of $1.05$. Performance explains $92.4\%$ of robustness variance, indicating a strong predictive relationship. Colors denote models, and shapes represent datasets. Dashed gray line: random baseline, i.e., the probability of answering consistently across all example configurations, assuming per-configuration success probability equals the model’s performance.
• Figure 2: Robustness rate per dataset, computed using the output consistency metric, in relation to overall performance (represented by dashed lines). Robustness rises as benchmarks saturate. Camel bars represent the random baseline consistency probability with similar success rates, which is approximately $0$ when performance is below $80\%$. Additional robustness metrics analysis can be found in App. §\ref{['sec:append_results']}.
• Figure 3: Average PDR reveals a similar trend: lower values indicate smaller performance drops across perturbations, signaling higher robustness and more consistent model behavior. The dashed lines indicates the performance.
• Figure 4: Per-example STD distribution for each model on the IMDb dataset.
• Figure 5: Per-example STD distribution for each model on the BoolQ dataset.