Zooming into Comics: Region-Aware RL Improves Fine-Grained Comic Understanding in Vision-Language Models

TL;DR

This work introduces AI4VA-FG, the first fine-grained benchmark for comic understanding that spans low-level recognition to high-level narrative reasoning with dense annotations. It evaluates both post-training strategies, namely supervised fine-tuning (SFT) and reinforcement learning (RL), and proposes Region-Aware Reinforcement Learning (RARL) to enable dynamic zoom-in reasoning on relevant regions. Empirical results show that RL and especially RARL improve performance on key tasks such as depth comparison and action recognition, and can narrow the gap toward larger proprietary models. The findings highlight persistent challenges in depth perception, character tracking, and multi-panel storytelling, while demonstrating that targeted region-grounded RL can enhance efficiency and effectiveness in comics understanding. This benchmark and method set the stage for more robust multimodal reasoning in dense visual narratives.

Abstract

Complex visual narratives, such as comics, present a significant challenge to Vision-Language Models (VLMs). Despite excelling on natural images, VLMs often struggle with stylized line art, onomatopoeia, and densely packed multi-panel layouts. To address this gap, we introduce AI4VA-FG, the first fine-grained and comprehensive benchmark for VLM-based comic understanding. It spans tasks from foundational recognition and detection to high-level character reasoning and narrative construction, supported by dense annotations for characters, poses, and depth. Beyond that, we evaluate state-of-the-art proprietary models, including GPT-4o and Gemini-2.5, and open-source models such as Qwen2.5-VL, revealing substantial performance deficits across core tasks of our benchmarks and underscoring that comic understanding remains an unsolved challenge. To enhance VLMs' capabilities in this domain, we systematically investigate post-training strategies, including supervised fine-tuning on solutions (SFT-S), supervised fine-tuning on reasoning trajectories (SFT-R), and reinforcement learning (RL). Beyond that, inspired by the emerging "Thinking with Images" paradigm, we propose Region-Aware Reinforcement Learning (RARL) for VLMs, which trains models to dynamically attend to relevant regions through zoom-in operations. We observe that when applied to the Qwen2.5-VL model, RL and RARL yield significant gains in low-level entity recognition and high-level storyline ordering, paving the way for more accurate and efficient VLM applications in the comics domain.

Figures (9)

• Figure 1: Benchmark Overview. (a) We introduce AI4VA-FG, a comic-centric benchmark featuring full-page, long-form narratives designed to challenge modern Vision-Language Models (VLMs). (b) The benchmark includes seven distinct tasks that evaluate a range of capabilities, from foundational recognition and detection to high-level plot and character understanding. (c) Our analysis shows that state-of-the-art VLMs struggle, but their performance is significantly boosted by post-training techniques, especially our proposed Region-Aware Reinforcement Learning.
• Figure 2: Performance of state-of-the-art open-source and proprietary models on AI4VA-FG. Although proprietary models achieve strong accuracy on most tasks, their performance remains inconsistent across the benchmark. Open-source models, in contrast, exhibit a 10–30 percentage point deficit, with the most pronounced weaknesses in depth perception, character tracking, and narrative construction.
• Figure 3: Statistics of our AI4VA-FG benchmark.
• Figure 4: VQA instances of common failures: (1) Depth Comparison (2) Character Identification (3) Panel Reordering.
• Figure 5: The full page image containing VQA instances shown in Fig.\ref{['fig:AI4VA-FG']}.