Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model

Tianle Li, Jihai Zhang, Yongming Rao, Yu Cheng

TL;DR

This work addresses the challenge of compositional reasoning in vision-language models under post-training regimes. It introduces ComPABench, a diagnostic benchmark that pairs pure-text and multimodal tasks with controlled distribution shifts to probe cross-modal, cross-task, and OOD generalization. The study finds that RL-based post-training more effectively integrates independently learned skills than supervised fine-tuning, but substantial gaps remain in multimodal compositional reasoning; a caption-before-thinking prompt with progressive grounding (RL-Ground) yields the strongest gains in both in-domain and OOD settings. These findings offer actionable strategies for building VLMs that reason compositionally across modalities and tasks, with implications for safe and scalable multimodal AI systems.

Abstract

While large language models (LLMs) demonstrate strong reasoning capabilities utilizing reinforcement learning (RL) with verifiable reward, whether large vision-language models (VLMs) can directly inherit such capabilities through similar post-training strategies remains underexplored. In this work, we conduct a systematic compositional probing study to evaluate whether current VLMs trained with RL or other post-training strategies can compose capabilities across modalities or tasks under out-of-distribution conditions. We design a suite of diagnostic tasks that train models on unimodal tasks or isolated reasoning skills, and evaluate them on multimodal, compositional variants requiring skill integration. Through comparisons between supervised fine-tuning (SFT) and RL-trained models, we identify three key findings: (1) RL-trained models consistently outperform SFT on compositional generalization, demonstrating better integration of learned skills; (2) although VLMs achieve strong performance on individual tasks, they struggle to generalize compositionally under cross-modal and cross-task scenario, revealing a significant gap in current training strategies; (3) enforcing models to explicitly describe visual content before reasoning (e.g., caption-before-thinking), along with rewarding progressive vision-to-text grounding, yields notable gains. It highlights two essential ingredients for improving compositionality in VLMs: visual-to-text alignment and accurate visual grounding. Our findings shed light on the current limitations of RL-based reasoning VLM training and provide actionable insights toward building models that reason compositionally across modalities and tasks.

Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model

TL;DR

This work addresses the challenge of compositional reasoning in vision-language models under post-training regimes. It introduces ComPABench, a diagnostic benchmark that pairs pure-text and multimodal tasks with controlled distribution shifts to probe cross-modal, cross-task, and OOD generalization. The study finds that RL-based post-training more effectively integrates independently learned skills than supervised fine-tuning, but substantial gaps remain in multimodal compositional reasoning; a caption-before-thinking prompt with progressive grounding (RL-Ground) yields the strongest gains in both in-domain and OOD settings. These findings offer actionable strategies for building VLMs that reason compositionally across modalities and tasks, with implications for safe and scalable multimodal AI systems.

Abstract

While large language models (LLMs) demonstrate strong reasoning capabilities utilizing reinforcement learning (RL) with verifiable reward, whether large vision-language models (VLMs) can directly inherit such capabilities through similar post-training strategies remains underexplored. In this work, we conduct a systematic compositional probing study to evaluate whether current VLMs trained with RL or other post-training strategies can compose capabilities across modalities or tasks under out-of-distribution conditions. We design a suite of diagnostic tasks that train models on unimodal tasks or isolated reasoning skills, and evaluate them on multimodal, compositional variants requiring skill integration. Through comparisons between supervised fine-tuning (SFT) and RL-trained models, we identify three key findings: (1) RL-trained models consistently outperform SFT on compositional generalization, demonstrating better integration of learned skills; (2) although VLMs achieve strong performance on individual tasks, they struggle to generalize compositionally under cross-modal and cross-task scenario, revealing a significant gap in current training strategies; (3) enforcing models to explicitly describe visual content before reasoning (e.g., caption-before-thinking), along with rewarding progressive vision-to-text grounding, yields notable gains. It highlights two essential ingredients for improving compositionality in VLMs: visual-to-text alignment and accurate visual grounding. Our findings shed light on the current limitations of RL-based reasoning VLM training and provide actionable insights toward building models that reason compositionally across modalities and tasks.

Paper Structure

This paper contains 25 sections, 2 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Post-training for VLM Reasoning
  4. Generalization Probing for Training Strategies
  5. Preliminaries
  6. Supervised Fine-Tuning
  7. Reinforcement Learning with GRPO
  8. SFT-Initialized RL Training
  9. Experiments
  10. Benchmark for Probing of Compositional Ability
  11. Cross-Modal Compositional ability (RQ1).
  12. Cross-Task Compositional ability (RQ2).
  13. Compositional OOD Generalization (RQ3).
  14. Experiment Settings
  15. Evaluation Result
  16. ...and 10 more sections

Figures (7)

  • Figure 1: Demonstration of the tasks and partial results for probing of cross-modality and cross-task compositional ability.
  • Figure 2: Demonstration of proposed ComPABench for RQ1, RQ2, and RQ3.
  • Figure 3: Performance comparison when post-trained with pure-text and evaluated with either pure-text or image-format(multi-modal) questions.
  • Figure 4: Trend of multi-modal performance without and with pure-text training initialization.
  • Figure 5: Comparison of Results for compositional reasoning from independently acquired skills.
  • ...and 2 more figures