Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conv-CoA: Improving Open-domain Question Answering in Large Language Models via Conversational Chain-of-Action

Zhenyu Pan, Haozheng Luo, Manling Li, Han Liu

TL;DR

Conv-CoA introduces a unified framework for Open-domain Conversational Question Answering that addresses hallucination, reasoning, and retrieval gaps by coupling a dynamic Action Chain with a Contextual Knowledge Set and a resource-efficient Hopfield-based retriever. The approach uses systematic prompting to decompose questions into sub-questions, retrieves relevant data via web-querying and local knowledge sources, and verifies answers with Conv-MRFS to maintain faithfulness. Key contributions include the resource-efficient Hopfield retriever, the Conv-MRFS faithfulness metric, and the integrated AC-CKS workflow that updates across turns to reduce information overlap. Empirical results on TopiOCQA and QReCC show superior accuracy and efficiency over 23 baselines, indicating strong potential for scalable, faithful OCQA systems in real-world settings.

Abstract

We present a Conversational Chain-of-Action (Conv-CoA) framework for Open-domain Conversational Question Answering (OCQA). Compared with literature, Conv-CoA addresses three major challenges: (i) unfaithful hallucination that is inconsistent with real-time or domain facts, (ii) weak reasoning performance in conversational scenarios, and (iii) unsatisfying performance in conversational information retrieval. Our key contribution is a dynamic reasoning-retrieval mechanism that extracts the intent of the question and decomposes it into a reasoning chain to be solved via systematic prompting, pre-designed actions, updating the Contextual Knowledge Set (CKS), and a novel Hopfield-based retriever. Methodologically, we propose a resource-efficiency Hopfield retriever to enhance the efficiency and accuracy of conversational information retrieval within our actions. Additionally, we propose a conversational-multi-reference faith score (Conv-MRFS) to verify and resolve conflicts between retrieved knowledge and answers in conversations. Empirically, we conduct comparisons between our framework and 23 state-of-the-art methods across five different research directions and two public benchmarks. These comparisons demonstrate that our Conv-CoA outperforms other methods in both the accuracy and efficiency dimensions.

Conv-CoA: Improving Open-domain Question Answering in Large Language Models via Conversational Chain-of-Action

TL;DR

Conv-CoA introduces a unified framework for Open-domain Conversational Question Answering that addresses hallucination, reasoning, and retrieval gaps by coupling a dynamic Action Chain with a Contextual Knowledge Set and a resource-efficient Hopfield-based retriever. The approach uses systematic prompting to decompose questions into sub-questions, retrieves relevant data via web-querying and local knowledge sources, and verifies answers with Conv-MRFS to maintain faithfulness. Key contributions include the resource-efficient Hopfield retriever, the Conv-MRFS faithfulness metric, and the integrated AC-CKS workflow that updates across turns to reduce information overlap. Empirical results on TopiOCQA and QReCC show superior accuracy and efficiency over 23 baselines, indicating strong potential for scalable, faithful OCQA systems in real-world settings.

Abstract

We present a Conversational Chain-of-Action (Conv-CoA) framework for Open-domain Conversational Question Answering (OCQA). Compared with literature, Conv-CoA addresses three major challenges: (i) unfaithful hallucination that is inconsistent with real-time or domain facts, (ii) weak reasoning performance in conversational scenarios, and (iii) unsatisfying performance in conversational information retrieval. Our key contribution is a dynamic reasoning-retrieval mechanism that extracts the intent of the question and decomposes it into a reasoning chain to be solved via systematic prompting, pre-designed actions, updating the Contextual Knowledge Set (CKS), and a novel Hopfield-based retriever. Methodologically, we propose a resource-efficiency Hopfield retriever to enhance the efficiency and accuracy of conversational information retrieval within our actions. Additionally, we propose a conversational-multi-reference faith score (Conv-MRFS) to verify and resolve conflicts between retrieved knowledge and answers in conversations. Empirically, we conduct comparisons between our framework and 23 state-of-the-art methods across five different research directions and two public benchmarks. These comparisons demonstrate that our Conv-CoA outperforms other methods in both the accuracy and efficiency dimensions.
Paper Structure (30 sections, 5 equations, 1 figure, 4 tables)

This paper contains 30 sections, 5 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Retrieval Phase
  4. Query Reformulation Methods
  5. Enhanced Retrieval Methods
  6. Prompting Methods
  7. Modern Hopfield Models and Hopfield Memory
  8. Methodology
  9. Task Definition
  10. Overview
  11. Contextual Knowledge Set
  12. Action chain generation
  13. Action Implementation
  14. Action design
  15. Web-querying.
  16. ...and 15 more sections

Figures (1)

  • Figure 1: Overview of Conv-CoA. It starts by injecting the initial question into a prompt, creating an action chain (AC) via the LLM. Each node in the AC represents a sub-question and an initial answer. Specific actions verify if the initial answer needs modification based on retrieved data. If the confidence in the initial answer is lower than in the retrieved data, an action prompts a change. The AC contents are stored in a contextual knowledge set (CKS) and updated at the end of each turn. In future turns, the CKS and the current question are combined to regenerate the AC, using different prompt templates to generate sub-questions for missing content, thus reducing information overlap.