GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Models

TL;DR

GGBench addresses the lack of integrated evaluative benchmarks for geometric generative reasoning in Unified Multimodal Models by introducing a tri-modal, verifiable benchmark that combines textual prompts, executable GeoGebra code, and rendered diagrams. It employs a rigorous data-collection and filtering pipeline to yield 1,411 high-quality construction problems, each with multiple images and aligned text/code/diagram triplets. An end-to-end evaluation across two tracks reveals a persistent gap: code-based, plan-to-construction pipelines achieve substantially higher geometric correctness and verifiability than pure image-generation approaches, highlighting the importance of executable grounding. The work demonstrates that verifiable, construction-centric evaluation offers a robust framework for advancing multimodal reasoning, with implications beyond geometry toward integrated language, logic, and construction assessment in AI systems.

Abstract

The advent of Unified Multimodal Models (UMMs) signals a paradigm shift in artificial intelligence, moving from passive perception to active, cross-modal generation. Despite their unprecedented ability to synthesize information, a critical gap persists in evaluation: existing benchmarks primarily assess discriminative understanding or unconstrained image generation separately, failing to measure the integrated cognitive process of generative reasoning. To bridge this gap, we propose that geometric construction provides an ideal testbed as it inherently demands a fusion of language comprehension and precise visual generation. We introduce GGBench, a benchmark designed specifically to evaluate geometric generative reasoning. It provides a comprehensive framework for systematically diagnosing a model's ability to not only understand and reason but to actively construct a solution, thereby setting a more rigorous standard for the next generation of intelligent systems. Project website: https://opendatalab-raiser.github.io/GGBench/.

Figures (22)

• Figure 1: The paradigm shift to generative reasoning. Conventional benchmarks evaluate (a) Understanding or (b) Generation in isolation. GGBench introduces (c) integrated Understanding& Generation evaluation, requiring generative reasoning from Unified Multimodal Models.
• Figure 2: GGBench's step-by-step evaluation. Beyond traditional text-image pairs, GGBench provides executable code for each construction step, allowing for precise and automated verification.
• Figure 3: Overview of the GGBench data construction pipeline.
• Figure 4: Difficulty distribution and category composition in GGBench. The inner ring shows the proportion of difficulty levels (Easy/Medium/Hard), while the outer ring presents category shares within each difficulty band, reflecting progressive complexity across reasoning types.
• Figure 5: VLM-I scores across eight construction categories in GGBench. Each cell reflects the average multimodal reasoning quality for a model-category pair.