VisualSphinx: Large-Scale Synthetic Vision Logic Puzzles for RL

TL;DR

VisualSphinx delivers a scalable, rule-grounded, large-scale synthetic dataset of visual logic puzzles to enhance vision-language reasoning through RL. Its four-stage pipeline—seed-rule abstraction, rule-level genetic expansion, programmatic image synthesis, and diversified puzzle assembly—enables cost-efficient generation (~$1K) of 660K puzzles. Fine-tuning Qwen2.5-VL-7B with GRPO on VisualSphinx yields notable improvements in visual logical reasoning and transfers to algebraic, arithmetic, and geometric tasks, as evidenced on MathVista. The work highlights the potential of synthetic multimodal data to boost robust, generalizable reasoning in VLMs, with implications for broader multimodal problem solving.

Abstract

Vision language models (VLMs) are expected to perform effective multimodal reasoning and make logically coherent decisions, which is critical to tasks such as diagram understanding and spatial problem solving. However, current VLM reasoning lacks large-scale and well-structured training datasets. To bridge this gap, we propose VisualSphinx, a first-of-its-kind large-scale synthetic visual logical reasoning training data. To tackle the challenge of image synthesis with grounding answers, we propose a rule-to-image synthesis pipeline, which extracts and expands puzzle rules from seed questions and generates the code of grounding synthesis image synthesis for puzzle sample assembly. Experiments demonstrate that VLM trained using GRPO on VisualSphinx benefit from logical coherence and readability of our dataset and exhibit improved performance on logical reasoning tasks. The enhanced reasoning capabilities developed from VisualSphinx also benefit other reasoning tasks such as algebraic reasoning, arithmetic reasoning and geometry reasoning.

Figures (28)

• Figure 1: Example VisualSphinx instances within each reasoning category. Each visual logic puzzle comprises a text prompt, a graphical question stem with four images and a question mark, and four candidate choices of graphical answers.
• Figure 2: This figure illustrates the four-stage pipeline for generating VisualSphinx. In Step 1, we collect 4K seed puzzles with explanations and abstract them into structured rule descriptions using LLMs. In Step 2, we apply a rule-level genetic algorithm to cross over, mutate and diversify the seed rules, scaling them to 40K high-quality rules. In Step 3, each rule is paired with a rendering style and used to generate five correct and three incorrect images via LLM-generated Python scripts. The fifth correct image is designated as the answer option, while the three rule-breaking images serve as distractors. After deduplication, we obtain 110K image groups. In Step 4, we assemble puzzles from each group using three complementary strategies: default assembly, shuffled answer variants, and expanded distractor sets. This results in over 660K visual logic puzzles, enabling robust and diverse training for multimodal reasoning models.
• Figure 3: Comparison of t-SNE visualization between VisualSphinx's synthetic rules and seed rules with 2,000 sampled rules per class. Details about 8 classes are provided in Appendix \ref{['appendix: classification']}.
• Figure 4: The statistics of synthetic rules format, content quality and feasibility score.
• Figure 5: This figure demonstrates the statistics of readability and logical coherence.