ProactiveBench: Benchmarking Proactiveness in Multimodal Large Language Models

Abstract

Effective collaboration begins with knowing when to ask for help. For example, when trying to identify an occluded object, a human would ask someone to remove the obstruction. Can MLLMs exhibit a similar "proactive" behavior by requesting simple user interventions? To investigate this, we introduce ProactiveBench, a benchmark built from seven repurposed datasets that tests proactiveness across different tasks such as recognizing occluded objects, enhancing image quality, and interpreting coarse sketches. We evaluate 22 MLLMs on ProactiveBench, showing that (i) they generally lack proactiveness; (ii) proactiveness does not correlate with model capacity; (iii) "hinting" at proactiveness yields only marginal gains. Surprisingly, we found that conversation histories and in-context learning introduce negative biases, hindering performance. Finally, we explore a simple fine-tuning strategy based on reinforcement learning: its results suggest that proactiveness can be learned, even generalizing to unseen scenarios. We publicly release ProactiveBench as a first step toward building proactive multimodal models.

Figures (21)

• Figure 1: Reactive v.s. proactive models. We propose ProactiveBench, the first benchmark to evaluate MLLMs' proactiveness, i.e., their ability to request additional visual cues to resolve ambiguous queries. Given an unanswerable query, a reactive model would either abstain or hallucinate. In contrast, a proactive model would ask for visual cues to disambiguate the input, enabling a correct response.
• Figure 2: ProactiveBench overview. ProactiveBench evaluates proactiveness in seven scenarios. The image shows examples of different scenarios and data statistics.
• Figure 3: Acc. in ProactiveBench vs. reference. Models underperform by over 60% in scenarios that require proactiveness.
• Figure 4: Action distributions. While LLaVA-OV-7B, InternVL3-8B, and LLaVA-NeXT-Mistral-7B abstain or guess an answer, the other models prioritize proactive suggestions; thus, leveraging better visual cues and making better predictions.
• Figure 5: Action distributions with random proactive options. Lighter bars describe variations when replacing valid proactive suggestions with invalid ones. We color-code positive and negative changes in action prob. If models still assign high prob. with random proactive actions, it implies they are not proactive and just avoid abstention.