Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Divide-and-Conquer: Tree-structured Strategy with Answer Distribution Estimator for Goal-Oriented Visual Dialogue

Shuo Cai, Xinzhe Han, Shuhui Wang

TL;DR

Goal-oriented visual dialogue suffers from unstructured questioning and high question repetition, hindering efficient target localization. The authors propose TSADE, a plug-in framework that uses a Tree-structured Strategy and an Answer Distribution Estimator to guide question generation by dividing candidate objects and maximizing information gain, with two RL rewards (binary and candidate-minimization). They show that TSADE reduces search complexity from $O(N)$ to $O(\log N)$ in ideal conditions and improves end-to-end accuracy and dialogue quality on GuessWhat?! and VisDial, across multiple baselines, including LVLM-based experiments. The work offers a practical, generalizable method to make goal-oriented visual dialogue more efficient and informative, with potential extensions toward embodied AI applications.

Abstract

Goal-oriented visual dialogue involves multi-round interaction between artificial agents, which has been of remarkable attention due to its wide applications. Given a visual scene, this task occurs when a Questioner asks an action-oriented question and an Answerer responds with the intent of letting the Questioner know the correct action to take. The quality of questions affects the accuracy and efficiency of the target search progress. However, existing methods lack a clear strategy to guide the generation of questions, resulting in the randomness in the search process and inconvergent results. We propose a Tree-Structured Strategy with Answer Distribution Estimator (TSADE) which guides the question generation by excluding half of the current candidate objects in each round. The above process is implemented by maximizing a binary reward inspired by the ``divide-and-conquer'' paradigm. We further design a candidate-minimization reward which encourages the model to narrow down the scope of candidate objects toward the end of the dialogue. We experimentally demonstrate that our method can enable the agents to achieve high task-oriented accuracy with fewer repeating questions and rounds compared to traditional ergodic question generation approaches. Qualitative results further show that TSADE facilitates agents to generate higher-quality questions.

Divide-and-Conquer: Tree-structured Strategy with Answer Distribution Estimator for Goal-Oriented Visual Dialogue

TL;DR

Goal-oriented visual dialogue suffers from unstructured questioning and high question repetition, hindering efficient target localization. The authors propose TSADE, a plug-in framework that uses a Tree-structured Strategy and an Answer Distribution Estimator to guide question generation by dividing candidate objects and maximizing information gain, with two RL rewards (binary and candidate-minimization). They show that TSADE reduces search complexity from to in ideal conditions and improves end-to-end accuracy and dialogue quality on GuessWhat?! and VisDial, across multiple baselines, including LVLM-based experiments. The work offers a practical, generalizable method to make goal-oriented visual dialogue more efficient and informative, with potential extensions toward embodied AI applications.

Abstract

Goal-oriented visual dialogue involves multi-round interaction between artificial agents, which has been of remarkable attention due to its wide applications. Given a visual scene, this task occurs when a Questioner asks an action-oriented question and an Answerer responds with the intent of letting the Questioner know the correct action to take. The quality of questions affects the accuracy and efficiency of the target search progress. However, existing methods lack a clear strategy to guide the generation of questions, resulting in the randomness in the search process and inconvergent results. We propose a Tree-Structured Strategy with Answer Distribution Estimator (TSADE) which guides the question generation by excluding half of the current candidate objects in each round. The above process is implemented by maximizing a binary reward inspired by the ``divide-and-conquer'' paradigm. We further design a candidate-minimization reward which encourages the model to narrow down the scope of candidate objects toward the end of the dialogue. We experimentally demonstrate that our method can enable the agents to achieve high task-oriented accuracy with fewer repeating questions and rounds compared to traditional ergodic question generation approaches. Qualitative results further show that TSADE facilitates agents to generate higher-quality questions.

Paper Structure

This paper contains 20 sections, 3 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Question Generator (QGen)
  4. Answer Distribution Estimator (ADE)
  5. Method
  6. Background
  7. Tree-structured Strategy with Answer Distribution Estimator (TSADE)
  8. Applicability in Two Settings
  9. Experiments
  10. Dataset
  11. Evaluation Metrics
  12. Implementation Details
  13. Comparison of Question Repetition Rate
  14. Comparison on Efficiency
  15. Comparison on Success Rate of End-to-end
  16. ...and 5 more sections

Figures (3)

  • Figure 1: (a) Illustration of goal-oriented visual dialogue. The target object is highlighted in green box. (b) Example of traditional dialogue strategy. (c) Example of Tree-structure dialogue strategy. The excluded objects are in the lower-right candidate box.
  • Figure 2: The framework of the Tree-structured Strategy with Answer Distribution Estimator (TSADE). The red box represents the target object.
  • Figure 3: The generated dialogue examples show the strategy for question generation with and without TSADE.