Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Few-Shot Change Detection Visual Question Answering via Decision-Ambiguity-guided Reinforcement Fine-Tuning

Fuyu Dong, Ke Li, Di Wang, Nan Luo, Yiming Zhang, Kaiyu Li, Jianfei Yang, Quan Wang

TL;DR

This paper tackles CDVQA by identifying decision ambiguity—where correct and strong distractor answers receive similar confidence—as a principal source of failures. It introduces Decision-Ambiguous Samples (DAS) andDARFT, a two-stage training approach that first aligns with supervised fine-tuning and then uses group-relative policy optimization to refine predictions on ambiguous cases. By mining DAS and optimizing relative advantages within answer groups, DARFT sharpens decision boundaries without additional labels, yielding robust gains, particularly in few-shot settings. The results on QAG-360K show consistent improvements over SFT baselines, including enhanced performance under multi-sample decoding, which indicates improved discrimination and decision stability in challenging CDVQA scenarios.

Abstract

Change detection visual question answering (CDVQA) requires answering text queries by reasoning about semantic changes in bi-temporal remote sensing images. A straightforward approach is to boost CDVQA performance with generic vision-language models via supervised fine-tuning (SFT). Despite recent progress, we observe that a significant portion of failures do not stem from clearly incorrect predictions, but from decision ambiguity, where the model assigns similar confidence to the correct answer and strong distractors. To formalize this challenge, we define Decision-Ambiguous Samples (DAS) as instances with a small probability margin between the ground-truth answer and the most competitive alternative. We argue that explicitly optimizing DAS is crucial for improving the discriminability and robustness of CDVQA models. To this end, we propose DARFT, a Decision-Ambiguity-guided Reinforcement Fine-Tuning framework that first mines DAS using an SFT-trained reference policy and then applies group-relative policy optimization on the mined subset. By leveraging multi-sample decoding and intra-group relative advantages, DARFT suppresses strong distractors and sharpens decision boundaries without additional supervision. Extensive experiments demonstrate consistent gains over SFT baselines, particularly under few-shot settings.

Improving Few-Shot Change Detection Visual Question Answering via Decision-Ambiguity-guided Reinforcement Fine-Tuning

TL;DR

This paper tackles CDVQA by identifying decision ambiguity—where correct and strong distractor answers receive similar confidence—as a principal source of failures. It introduces Decision-Ambiguous Samples (DAS) andDARFT, a two-stage training approach that first aligns with supervised fine-tuning and then uses group-relative policy optimization to refine predictions on ambiguous cases. By mining DAS and optimizing relative advantages within answer groups, DARFT sharpens decision boundaries without additional labels, yielding robust gains, particularly in few-shot settings. The results on QAG-360K show consistent improvements over SFT baselines, including enhanced performance under multi-sample decoding, which indicates improved discrimination and decision stability in challenging CDVQA scenarios.

Abstract

Change detection visual question answering (CDVQA) requires answering text queries by reasoning about semantic changes in bi-temporal remote sensing images. A straightforward approach is to boost CDVQA performance with generic vision-language models via supervised fine-tuning (SFT). Despite recent progress, we observe that a significant portion of failures do not stem from clearly incorrect predictions, but from decision ambiguity, where the model assigns similar confidence to the correct answer and strong distractors. To formalize this challenge, we define Decision-Ambiguous Samples (DAS) as instances with a small probability margin between the ground-truth answer and the most competitive alternative. We argue that explicitly optimizing DAS is crucial for improving the discriminability and robustness of CDVQA models. To this end, we propose DARFT, a Decision-Ambiguity-guided Reinforcement Fine-Tuning framework that first mines DAS using an SFT-trained reference policy and then applies group-relative policy optimization on the mined subset. By leveraging multi-sample decoding and intra-group relative advantages, DARFT suppresses strong distractors and sharpens decision boundaries without additional supervision. Extensive experiments demonstrate consistent gains over SFT baselines, particularly under few-shot settings.
Paper Structure (13 sections, 15 equations, 2 figures, 3 tables, 1 algorithm)

This paper contains 13 sections, 15 equations, 2 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methodology
  3. Problem Formulation
  4. Model Interface and Output Space
  5. Stage I: Task Alignment via Supervised Fine-Tuning
  6. Decision-Ambiguous Sample Mining
  7. Stage II: Decision-Ambiguity-Guided GRPO
  8. Inference
  9. Experiments
  10. Experimental Setup
  11. Main Results
  12. Decision Stability under Multi-sample Inference.
  13. Conclusion

Figures (2)

  • Figure 1: Decision ambiguity distribution before and after DARFT. (a) Histogram of decision ambiguity scores $\Delta(x)$, where $\Delta(x)$ denotes the probability margin between the ground-truth answer and the strongest competing distractor. (b) Corresponding cumulative distribution functions of $\Delta(x)$. Compared to SFT, the proposed DARFT substantially shifts the distribution toward larger margins, indicating reduced decision ambiguity and sharper decision boundaries.
  • Figure 2: The visualization of the training pipeline of our DARFT framework.