On Language Models' Sensitivity to Suspicious Coincidences

TL;DR

The paper investigates whether language models exhibit suspicious coincidences in inductive generalization, using the Bayesian size principle as a reference. It analyzes two domains, the Number game and City game, across five LMs with zero-shot, chain-of-thought, and knowledge-based prompting. Zero-shot LMs show little alignment with the smallest-hypothesis bias, but prompting that exposes the hypothesis space (especially Knowledge prompts) yields human-like sensitivity and, for GPT-4o, near-Bayesian performance in the number domain. The results imply that incorporating explicit hypothesis-landscape access can substantially enhance inductive reasoning in LMs, with practical implications for designing prompts or distillation methods to foster principled generalization.

Abstract

Humans are sensitive to suspicious coincidences when generalizing inductively over data, as they make assumptions as to how the data was sampled. This results in smaller, more specific hypotheses being favored over more general ones. For instance, when provided the set {Austin, Dallas, Houston}, one is more likely to think that this is sampled from "Texas Cities" over "US Cities" even though both are compatible. Suspicious coincidence is strongly connected to pragmatic reasoning, and can serve as a testbed to analyze systems on their sensitivity towards the communicative goals of the task (i.e., figuring out the true category underlying the data). In this paper, we analyze whether suspicious coincidence effects are reflected in language models' (LMs) behavior. We do so in the context of two domains: 1) the number game, where humans made judgments of whether a number (e.g., 4) fits a list of given numbers (e.g., 16, 32, 2); and 2) by extending the number game setup to prominent cities. For both domains, the data is compatible with multiple hypotheses and we study which hypothesis is most consistent with the models' behavior. On analyzing five models, we do not find strong evidence for suspicious coincidences in LMs' zero-shot behavior. However, when provided access to the hypotheses space via chain-of-thought or explicit prompting, LMs start to show an effect resembling suspicious coincidences, sometimes even showing effects consistent with humans. Our study suggests that inductive reasoning behavior in LMs can be enhanced with explicit access to the hypothesis landscape.

Figures (3)

• Figure 1: Compatibility of predictions with two hypothesis (Odd, End3) from five systems: random baseline, GPT-4o (0-shot), GPT-4o (with access to hypotheses information, denoted as 'Knowledge'), Bayesian model and human predictions. When suspicious coincidences is observed {93, 43, 83, 53}, three systems (Humans, Bayesian model, GPT-4o (Knowledge) strongly favors smaller hypothesis (End3), while other systems do not.
• Figure 2: Data statistics.
• Figure 4: Percentage of time models/humans prefer the smallest hypothesis and the Avg. F1 of the smallest hypotheses with the model/human predictions, as a function of input size, for Numbers and Cities. In both cases, an increase in the respective metric as the input size increases is indicative of a suspicious coincidence effect. Baselines and Comparison systems are under the 'Comparison' column, and the five LLMs along with the three prompting methods are shown as separate columns. Error bars indicate 95% confidence intervals.