MM-HELIX: Boosting Multimodal Long-Chain Reflective Reasoning with Holistic Platform and Adaptive Hybrid Policy Optimization

TL;DR

This work addresses the lack of long-chain multimodal reflective reasoning in current MLLMs by introducing MM-HELIX, a 1,260-sample benchmark of 42 multimodal tasks across four domains. It then presents MM-HELIX-100K, a large-scale CoT dataset generated via the Step-Elicited Response Generation pipeline to guide instruction tuning, and AHPO, an Adaptive Hybrid Policy Optimization method that unifies offline supervision with online exploration in a single stage. AHPO leverages high-quality CoT trajectories from MM-HELIX-100K and demonstrates substantial gains on MM-HELIX (+18.6 points) and transfer to general mathematics and logic tasks (+5.7 points), while SERG improves data efficiency and CoT quality compared to direct LLM rollouts or rule-based generation. The results indicate reflective reasoning is a transferable competence for MLLMs, paving the way towards more capable multimodal reasoning systems. The proposed framework supports robust evaluation and training for challenging, real-world reasoning that integrates perception and complex rule-based tasks.

Abstract

While current Multimodal Large Language Models (MLLMs) have demonstrated proficiency in reasoning tasks such as mathematics and logic, their capacity for long-chain reflective reasoning, a prerequisite for solving complex real-world problems, remains largely underexplored. In this work, we first conduct an extensive empirical investigation to evaluate this capability. Leveraging a carefully designed data synthesis engine, we construct MM-HELIX, a multimodal benchmark consisting 1,260 samples of 42 challenging synthetic tasks that require iterative thinking and backtracking. Empirical results on this benchmark reveal that existing MLLMs exhibit significant performance deficits in long-chain reflective reasoning. To address this limitation, we generate post-training data and further explore learning paradigms for exploiting such data. We first develop the Step-Elicited Response Generation pipeline to create MM-HELIX-100K, a large-scale dataset of 100k high-quality, reflective reasoning traces for instruction-tuning stage. Given that standard Reinforcement Learning fails on complex tasks due to sparse reward signals and catastrophic forgetting after Supervised Fine-Tuning, we propose Adaptive Hybrid Policy Optimization (AHPO), a novel training strategy that dynamically unifies offline supervision and online optimization into a single stage. This strategy enables the model to learn from expert data when rewards are sparse and conduct independent exploration once proficient. When applied to the Qwen2.5-VL-7B baseline, our method achieves a +18.6\% accuracy improvement on MM-HELIX benchmark and demonstrates strong generalization with a +5.7\% average performance gain on general mathematic and logic tasks. Our work demonstrate that reflective reasoning in MLLMs can be effectively learned and generalized, paving the way for developing more capable MLLMs.

Figures (8)

• Figure 1: Overview of proposed framework. Our framework comprises two core components: (1) MM-HELIX benchmark to evaluate the reflective capabilities of MLLM, and (2) AHPO method to boost reflection capability and transfer enhanced skills to general reasoning tasks.
• Figure 2: Overview of tasks in MM-HELIX benchmark. MM-HELIX contains 42 challenging tasks designed to evaluate long-chain reflective reasoning across five progressive levels of difficulty.
• Figure 3: Example of Nibbles task (Level 5) in MM-HELIX benchmark. The snake must eat all apples on the grid by executing a sequence of moves, demanding long-term reflection.
• Figure 4: Demonstration of our Step-Elicited Response Generation pipeline.
• Figure 5: Demonstration of Adaptive Hybrid Policy Optimization (AHPO). AHPO dynamically integrates off-policy expert guidance with on-policy exploration, leading to performance generalization.