Learning Through Little Eyes: Attribute Discrimination Beyond Objects

TL;DR

This work investigates whether infant-scale vision–language models can discriminate fine-grained visual attributes within object categories. It introduces a controlled benchmark that varies color, size, and texture using synthetic images across Konkle categories, and evaluates CVCL and CLIP on within-class attribute tasks. Results show CVCL excels at size discrimination while CLIP grounds color well, with both models displaying texture representations in image space but limited linguistic grounding. The findings reveal distinct inductive biases from different training regimes and provide a framework for probing attribute-level representations to guide future hybrid training approaches.

Abstract

Infants learn to recognize not only object categories but also fine grained attributes such as color, size, and texture within their first two years of life. Prior work explores Childs View for Contrastive Learning (CVCL), a CLIP style model trained on infant egocentric video as a computational model of early infant learning, but it focuses only on class level recognition. This leaves it unclear whether infant scale learning also supports attribute discrimination. To address this, we introduce a benchmark that systematically varies color, size, and texture, allowing controlled tests of within class attribute recognition. Comparing CVCL with CLIP shows clear differences. CVCL is better at size discrimination, while CLIP achieves higher accuracy on color discrimination. Both models represent texture in image embeddings but fail to ground texture linguistically, suggesting a gap between visual and language spaces.

Figures (6)

• Figure 1: (a, b): Infants are efficient learners, which has inspired research under similar circumstances; prior work vong2024grounded trained a CLIP-like model using infants' egocentric views and transcribed parental speech. However, their testing is limited to object category discrimination. (c–f): We benchmark the model with infant-level attribute discrimination, controlling color, size, and texture without changing other visual characteristics.
• Figure 2: Illustration of the text-encoder (image–text) test: a query (“blue ball”) is matched against candidates of the same class differing only in color.
• Figure 3: Per-class classification accuracy in prototype mode. CVCL shows selective strengths in infant-relevant categories, but CLIP dominates overall.
• Figure 4: Per-class classification accuracy in text--vision mode. CLIP achieves high performance across categories, whereas CVCL remains near chance.
• Figure 5: Attribute discrimination in prototype (image-only) mode. CVCL excels at size, CLIP dominates color, and both succeed on texture.