Bears, all bears, and some bears. Language Constraints on Language Models' Inductive Inferences

TL;DR

This work investigates how subtle surface-form cues modulate inductive inferences in Vision-Language Models, mirroring developmental findings that all, generics, and some constrain generalization differently. Through a multi-stage program (category identification, all-some sensitivity, and inductive constraint testing) across vision+language and language-only modalities, the study shows that selected VLMs exhibit human-like patterns, notably all > generics > some, and that their internal representations separate propositions by inductive constraints rather than surface form. The authors also provide a developmentally inspired, publicly released benchmark for all/some across modalities, and show that low-dimensional model representations reflect constraint-based organization emerging in mid-to-late layers. While offering evidence of alignment with human inductive reasoning, the work discusses limitations around causal interpretation, model-human differences, and the need for broader, multilingual benchmarking to deepen our understanding of generics in AI systems.

Abstract

Language places subtle constraints on how we make inductive inferences. Developmental evidence by Gelman et al. (2002) has shown children (4 years and older) to differentiate among generic statements ("Bears are daxable"), universally quantified NPs ("all bears are daxable") and indefinite plural NPs ("some bears are daxable") in extending novel properties to a specific member (all > generics > some), suggesting that they represent these types of propositions differently. We test if these subtle differences arise in general purpose statistical learners like Vision Language Models, by replicating the original experiment. On tasking them through a series of precondition tests (robust identification of categories in images and sensitivities to all and some), followed by the original experiment, we find behavioral alignment between models and humans. Post-hoc analyses on their representations revealed that these differences are organized based on inductive constraints and not surface-form differences.

Figures (9)

• Figure 1: We study how the surface form of a proposition constrains a model's inductive inferences. We compare between universal quantifiers (all), generics (bare plurals), and indefinite quantifiers (some). Humans represent them differently and show consistent graded effects in their inductive inferences gelman2002children.
• Figure 2: An instance from our category identification experiment. Each positive sample is associated with three separate types of negative samples.
• Figure 3: Accuracy of VLMs on all and some, across modalities. Each point represents the accuracy of a model on a subset of the dataset, except for 'Average', which denotes the average accuracy across subsets.
• Figure 4: Avg. probabilities of extending the property from the premise to the specific instance in the question (i.e., predicting Yes) in Qwen3-VL models (4B and 8B) for animate and inanimate categories, across both modalities, and premise types (all vs. generics vs. some). Error bars denote 95% confidence intervals, over prompt variations (templates and properties).
• Figure 5: First two Principal Components of the last hidden state representations in selected layers of Qwen3-VL-8B for stimuli that attribute properties to categories and vary in their scope, as modulated by quantifiers (all/every vs. certain/some) or generics (bare plural/indefinite). Results for all layers are shown in \ref{['fig:pca-full']}.