Reinforcement Learning for Conversational Question Answering over Knowledge Graph

TL;DR

The paper addresses the challenge of Conversational Question Answering over law knowledge graphs under noisy input by proposing an iterative reinforcement-learning framework. It combines a BERT-based question encoder with an LSTM history encoder to inform an MDP-based legal inquiry resolver that traverses a knowledge graph starting from a designated entity. The policy network selects graph edges via a structured embedding of relations, edges, and endpoints, and training uses REINFORCE to maximize cumulative rewards across dialogues. Experimental results on ConvQuestions and ConvRef show competitive to state-of-the-art performance, with particularly strong Hit@5 on ConvRef, demonstrating robustness to noisy, multi-turn legal queries and the value of history-aware navigation in KGQA. This approach advances practical, multi-turn legal QA by enabling autonomous, goal-directed graph exploration under uncertain user input.

Abstract

Conversational question answering (ConvQA) over law knowledge bases (KBs) involves answering multi-turn natural language questions about law and hope to find answers in the law knowledge base. Despite many methods have been proposed. Existing law knowledge base ConvQA model assume that the input question is clear and can perfectly reflect user's intention. However, in real world, the input questions are noisy and inexplict. This makes the model hard to find the correct answer in the law knowledge bases. In this paper, we try to use reinforcement learning to solve this problem. The reinforcement learning agent can automatically learn how to find the answer based on the input question and the conversation history, even when the input question is inexplicit. We test the proposed method on several real world datasets and the results show the effectivenss of the proposed model.