Reliable Conversational Agents under ASP Control that Understand Natural Language
Yankai Zeng
TL;DR
Large Language Models provide fluent conversational abilities but struggle with factual accuracy and strict adherence to data and rules, making them unreliable for knowledge-based tasks. The authors propose the STAR framework, which uses an LLM only as a parser to translate natural language into predicate-based knowledge, while an Answer Set Programming (ASP) reasoner handles consistency, completion, and inference over the knowledge. The approach is instantiated with AutoConcierge (task-oriented) and AutoCompanion (social chatbot) to demonstrate how predicate-grounded reasoning can guide dialogue and enable multi-turn interactions with verifiable outcomes. Preliminary results show GPT-3.5 achieving about 89% accuracy in NL-to-predicate translation on the E2E dataset, and case studies illustrating automatic querying and completion via CKTs and RCC-driven topic shifts. The work contributes a general, scalable blueprint for reliable, explainable conversational agents that can be trained and extended to new domains.
Abstract
Efforts have been made to make machines converse like humans in the past few decades. The recent techniques of Large Language Models (LLMs) make it possible to have human-like conversations with machines, but LLM's flaws of lacking understanding and reliability are well documented. We believe that the best way to eliminate this problem is to use LLMs only as parsers to translate text to knowledge and vice versa and carry out the conversation by reasoning over this knowledge using the answer set programming. I have been developing a framework based on LLMs and ASP to realize reliable chatbots that "understand" human conversation. This framework has been used to develop task-specific chatbots as well as socialbots. My future research is focused on making these chatbots scalable and trainable.