Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BabyVision: Visual Reasoning Beyond Language

Liang Chen, Weichu Xie, Yiyan Liang, Hongfeng He, Hans Zhao, Zhibo Yang, Zhiqi Huang, Haoning Wu, Haoyu Lu, Y. charles, Yiping Bao, Yuantao Fan, Guopeng Li, Haiyang Shen, Xuanzhong Chen, Wendong Xu, Shuzheng Si, Zefan Cai, Wenhao Chai, Ziqi Huang, Fangfu Liu, Tianyu Liu, Baobao Chang, Xiaobo Hu, Kaiyuan Chen, Yixin Ren, Yang Liu, Yuan Gong, Kuan Li

TL;DR

The paper identifies a fundamental gap in current Multimodal LLMs: while these models excel at knowledge-rich tasks, they struggle with core, pre-linguistic visual primitives that humans acquire early in life. It introduces BabyVision, a benchmark of 388 questions across four domains aimed at testing purely perceptual abilities with minimal linguistic priors, and BabyVision-Gen to assess visual reasoning through generation. Across 11 frontier models, results show a large human–model gap (roughly 44 points) and pronounced deficits in visual tracking, spatial perception, and detail preservation, revealing the verbalization bottleneck as a key barrier. The authors also explore RLVR as a training aid and demonstrate that generation-based visual reasoning and native visual externalization are promising directions for developing truly grounded visual intelligence in multimodal systems.

Abstract

While humans develop core visual skills long before acquiring language, contemporary Multimodal LLMs (MLLMs) still rely heavily on linguistic priors to compensate for their fragile visual understanding. We uncovered a crucial fact: state-of-the-art MLLMs consistently fail on basic visual tasks that humans, even 3-year-olds, can solve effortlessly. To systematically investigate this gap, we introduce BabyVision, a benchmark designed to assess core visual abilities independent of linguistic knowledge for MLLMs. BabyVision spans a wide range of tasks, with 388 items divided into 22 subclasses across four key categories. Empirical results and human evaluation reveal that leading MLLMs perform significantly below human baselines. Gemini3-Pro-Preview scores 49.7, lagging behind 6-year-old humans and falling well behind the average adult score of 94.1. These results show despite excelling in knowledge-heavy evaluations, current MLLMs still lack fundamental visual primitives. Progress in BabyVision represents a step toward human-level visual perception and reasoning capabilities. We also explore solving visual reasoning with generation models by proposing BabyVision-Gen and automatic evaluation toolkit. Our code and benchmark data are released at https://github.com/UniPat-AI/BabyVision for reproduction.

BabyVision: Visual Reasoning Beyond Language

TL;DR

The paper identifies a fundamental gap in current Multimodal LLMs: while these models excel at knowledge-rich tasks, they struggle with core, pre-linguistic visual primitives that humans acquire early in life. It introduces BabyVision, a benchmark of 388 questions across four domains aimed at testing purely perceptual abilities with minimal linguistic priors, and BabyVision-Gen to assess visual reasoning through generation. Across 11 frontier models, results show a large human–model gap (roughly 44 points) and pronounced deficits in visual tracking, spatial perception, and detail preservation, revealing the verbalization bottleneck as a key barrier. The authors also explore RLVR as a training aid and demonstrate that generation-based visual reasoning and native visual externalization are promising directions for developing truly grounded visual intelligence in multimodal systems.

Abstract

While humans develop core visual skills long before acquiring language, contemporary Multimodal LLMs (MLLMs) still rely heavily on linguistic priors to compensate for their fragile visual understanding. We uncovered a crucial fact: state-of-the-art MLLMs consistently fail on basic visual tasks that humans, even 3-year-olds, can solve effortlessly. To systematically investigate this gap, we introduce BabyVision, a benchmark designed to assess core visual abilities independent of linguistic knowledge for MLLMs. BabyVision spans a wide range of tasks, with 388 items divided into 22 subclasses across four key categories. Empirical results and human evaluation reveal that leading MLLMs perform significantly below human baselines. Gemini3-Pro-Preview scores 49.7, lagging behind 6-year-old humans and falling well behind the average adult score of 94.1. These results show despite excelling in knowledge-heavy evaluations, current MLLMs still lack fundamental visual primitives. Progress in BabyVision represents a step toward human-level visual perception and reasoning capabilities. We also explore solving visual reasoning with generation models by proposing BabyVision-Gen and automatic evaluation toolkit. Our code and benchmark data are released at https://github.com/UniPat-AI/BabyVision for reproduction.
Paper Structure (56 sections, 13 figures, 7 tables)

This paper contains 56 sections, 13 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Benchmarks for Multimodal Large Language Models
  4. Early Vision and Developmental Foundations
  5. BabyVision
  6. Data Curation Pipeline
  7. Taxonomy Definition and Seed Collection.
  8. Data Collection and Filtering.
  9. Annotation and Quality Assurance.
  10. Dataset Statistics
  11. Evaluation
  12. BabyVision-Gen
  13. Test Reasoning in Generation Models
  14. Statistics.
  15. Evaluation
  16. ...and 41 more sections

Figures (13)

  • Figure 1: Performance on BabyVision among MLLMs and human of different ages.
  • Figure 2: Fine-grained performance analysis on the full BabyVision benchmark.
  • Figure 3: Examples of BabyVision and BabyVision-Gen. While BabyVision evaluates visual understanding through language output, BabyVision-Gen evaluates visual reasoning through image generation.
  • Figure 4: Overview of the multi-stage data collection and curation pipeline for BabyVision: taxonomy design & seed selection, data augmentation & filtering, and annotation & quality assurance.
  • Figure 5: Example questions and the number of examples (#) from BabyVision spanning the four core categories and 22 types.
  • ...and 8 more figures