ProSLM : A Prolog Synergized Language Model for explainable Domain Specific Knowledge Based Question Answering

TL;DR

This work proposes a novel neurosymbolic framework that provides a domain-specific knowledge base, a logical reasoning system, and an integration to an existing LLM to improve the robustness and reliability of LLMs in question-answering tasks.

Abstract

Neurosymbolic approaches can add robustness to opaque neural systems by incorporating explainable symbolic representations. However, previous approaches have not used formal logic to contextualize queries to and validate outputs of large language models (LLMs). We propose \systemname{}, a novel neurosymbolic framework, to improve the robustness and reliability of LLMs in question-answering tasks. We provide \systemname{} with a domain-specific knowledge base, a logical reasoning system, and an integration to an existing LLM. This framework has two capabilities (1) context gathering: generating explainable and relevant context for a given query, and (2) validation: confirming and validating the factual accuracy of a statement in accordance with a knowledge base (KB). Our work opens a new area of neurosymbolic generative AI text validation and user personalization.

Figures (4)

• Figure 1: ProSLM system workflow diagram
• Figure 2: Flow of information diagram for ProSLM in an explainable context gathering scenario. The system flow begins with the user query in natural language Q(n). The Neural Translator converts Q(n) to a Prolog query Q(p), which is fed to the inference engine. The inference engine runs a backward chain with the Q(p) as the goal and returns the context C(p) in the form of Prolog output, creating an goal tree in the process. The Neural Translator converts this context C(p) to a natural language context C(n). The Neural Generator is then fed with user query Q(n), context C(n), and a prompt P which informs the LLM to utilize C(n) and respond with reasoning. The final result is a Response R that is grounded by the human-interpretable, explainable, and accurate context used to generate it.
• Figure 3: Goal tree for the query status(pool, 1100, monday, Y)
• Figure 4: Flow of information diagram for ProSLM in a fact checking scenario. The Neural Generator creates a response R based on the user query. This response R is fed into our Translator, which converts every fact in R into a list of Prolog queries. Every Prolog query is then be sent to our Symbolic Component for validation. For each query, the Symbolic Component returns True if that query is asserted by the KB, else False. These truth values are returned along with the original response R to the user.