Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spatial Preference Rewarding for MLLMs Spatial Understanding

Han Qiu, Peng Gao, Lewei Lu, Xiaoqin Zhang, Ling Shao, Shijian Lu

TL;DR

SPR tackles the mismatch between user spatial expectations and MLLM outputs by adding a self-contained preference-based signal for spatial reasoning. It constructs random region queries, computes a semantic score and a localization score, and uses Direct Preference Optimization with LoRA to train MLLMs to produce detailed, accurately localized region descriptions, including a grounding refinement step. The key contributions are an automated region-query/ranking pipeline, refinement of localization in preferred responses, and empirical gains across REC, grounding, and region-captioning benchmarks, with a training cost similar to standard instruction-tuning and a weighting scheme $S = \lambda S_{sem} + (1-\lambda) S_{loc}$ with $\lambda=0.8$. This work advances practical spatial grounding in MLLMs by reducing hallucinations and improving trustworthiness for real-world visual-language tasks.

Abstract

Multimodal large language models~(MLLMs) have demonstrated promising spatial understanding capabilities, such as referencing and grounding object descriptions. Despite their successes, MLLMs still fall short in fine-grained spatial perception abilities, such as generating detailed region descriptions or accurately localizing objects. Additionally, they often fail to respond to the user's requirements for desired fine-grained spatial understanding. This issue might arise because existing approaches primarily focus on tuning MLLMs to model pre-annotated instruction data to inject spatial knowledge, without direct supervision of MLLMs' actual responses. We address this issue by SPR, a Spatial Preference Rewarding~(SPR) approach that enhances MLLMs' spatial capabilities by rewarding MLLMs' detailed responses with precise object localization over vague or inaccurate responses. With randomly selected image regions and region descriptions from MLLMs, SPR introduces semantic and localization scores to comprehensively evaluate the text quality and localization quality in MLLM-generated descriptions. We also refine the MLLM descriptions with better localization accuracy and pair the best-scored refinement with the initial descriptions of the lowest score for direct preference optimization, thereby enhancing fine-grained alignment with visual input. Extensive experiments over standard referring and grounding benchmarks show that SPR improves MLLM spatial understanding capabilities effectively with minimal overhead in training. Data and code will be released at https://github.com/hanqiu-hq/SPR

Spatial Preference Rewarding for MLLMs Spatial Understanding

TL;DR

SPR tackles the mismatch between user spatial expectations and MLLM outputs by adding a self-contained preference-based signal for spatial reasoning. It constructs random region queries, computes a semantic score and a localization score, and uses Direct Preference Optimization with LoRA to train MLLMs to produce detailed, accurately localized region descriptions, including a grounding refinement step. The key contributions are an automated region-query/ranking pipeline, refinement of localization in preferred responses, and empirical gains across REC, grounding, and region-captioning benchmarks, with a training cost similar to standard instruction-tuning and a weighting scheme with . This work advances practical spatial grounding in MLLMs by reducing hallucinations and improving trustworthiness for real-world visual-language tasks.

Abstract

Multimodal large language models~(MLLMs) have demonstrated promising spatial understanding capabilities, such as referencing and grounding object descriptions. Despite their successes, MLLMs still fall short in fine-grained spatial perception abilities, such as generating detailed region descriptions or accurately localizing objects. Additionally, they often fail to respond to the user's requirements for desired fine-grained spatial understanding. This issue might arise because existing approaches primarily focus on tuning MLLMs to model pre-annotated instruction data to inject spatial knowledge, without direct supervision of MLLMs' actual responses. We address this issue by SPR, a Spatial Preference Rewarding~(SPR) approach that enhances MLLMs' spatial capabilities by rewarding MLLMs' detailed responses with precise object localization over vague or inaccurate responses. With randomly selected image regions and region descriptions from MLLMs, SPR introduces semantic and localization scores to comprehensively evaluate the text quality and localization quality in MLLM-generated descriptions. We also refine the MLLM descriptions with better localization accuracy and pair the best-scored refinement with the initial descriptions of the lowest score for direct preference optimization, thereby enhancing fine-grained alignment with visual input. Extensive experiments over standard referring and grounding benchmarks show that SPR improves MLLM spatial understanding capabilities effectively with minimal overhead in training. Data and code will be released at https://github.com/hanqiu-hq/SPR
Paper Structure (16 sections, 4 equations, 2 figures, 8 tables, 1 algorithm)

This paper contains 16 sections, 4 equations, 2 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Grounded Region Description Generation
  5. Preference Data Ranking and Construction
  6. Preference Optimization
  7. Experiments
  8. Experiment Setups.
  9. Experiments on REC
  10. Experiments on Phrase Grounding
  11. Experiments on Region Captioning
  12. Experiments on Ferret Bench
  13. Experiments on General Benchmarks
  14. Ablation Studies
  15. Comparison with SFT
  16. ...and 1 more sections

Figures (2)

  • Figure 1: The proposed Spatial Preference Rewarding (SPR) mitigates the distracted and inaccurate region descriptions generated by MLLMs. Given an image and a user-specified region of interest, MLLMs often fail to focus on the queried region. They may be distracted by objects outside the specified region, failing to ground the queried objects, or providing inaccurate localization. Tuning MLLMs with our proposed SPR leads to more accurate object localization and detailed object descriptions.
  • Figure 2: We leverage the generated object references and cropped image region to build a variety of multimodal prompts to enhance the diversity of generated region descriptions.