Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Building Trust in Conversational AI: A Comprehensive Review and Solution Architecture for Explainable, Privacy-Aware Systems using LLMs and Knowledge Graph

Ahtsham Zafar, Venkatesh Balavadhani Parthasarathy, Chan Le Van, Saad Shahid, Aafaq Iqbal khan, Arsalan Shahid

TL;DR

A novel functional architecture is proposed that seamlessly integrates the structured dynamics of knowledge graphs with the linguistic capabilities of LLMs and further strengthens data security through role-based access control.

Abstract

Conversational AI systems have emerged as key enablers of human-like interactions across diverse sectors. Nevertheless, the balance between linguistic nuance and factual accuracy has proven elusive. In this paper, we first introduce LLMXplorer, a comprehensive tool that provides an in-depth review of over 150 Large Language Models (LLMs), elucidating their myriad implications ranging from social and ethical to regulatory, as well as their applicability across industries. Building on this foundation, we propose a novel functional architecture that seamlessly integrates the structured dynamics of Knowledge Graphs with the linguistic capabilities of LLMs. Validated using real-world AI news data, our architecture adeptly blends linguistic sophistication with factual rigour and further strengthens data security through Role-Based Access Control. This research provides insights into the evolving landscape of conversational AI, emphasizing the imperative for systems that are efficient, transparent, and trustworthy.

Building Trust in Conversational AI: A Comprehensive Review and Solution Architecture for Explainable, Privacy-Aware Systems using LLMs and Knowledge Graph

TL;DR

A novel functional architecture is proposed that seamlessly integrates the structured dynamics of knowledge graphs with the linguistic capabilities of LLMs and further strengthens data security through role-based access control.

Abstract

Conversational AI systems have emerged as key enablers of human-like interactions across diverse sectors. Nevertheless, the balance between linguistic nuance and factual accuracy has proven elusive. In this paper, we first introduce LLMXplorer, a comprehensive tool that provides an in-depth review of over 150 Large Language Models (LLMs), elucidating their myriad implications ranging from social and ethical to regulatory, as well as their applicability across industries. Building on this foundation, we propose a novel functional architecture that seamlessly integrates the structured dynamics of Knowledge Graphs with the linguistic capabilities of LLMs. Validated using real-world AI news data, our architecture adeptly blends linguistic sophistication with factual rigour and further strengthens data security through Role-Based Access Control. This research provides insights into the evolving landscape of conversational AI, emphasizing the imperative for systems that are efficient, transparent, and trustworthy.
Paper Structure (33 sections, 13 figures, 10 tables, 3 algorithms)

This paper contains 33 sections, 13 figures, 10 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. History and Evolution of Conversational AI
  3. Development and Advancement of LLMs
  4. Main Contributions and Paper Organisation
  5. Methods and training process of LLMs
  6. Pretraining
  7. Supervised Fine-tuning
  8. Dialogue Optimization
  9. Reward Model
  10. Reinforcement Learning from Human Feedback (RLHF)
  11. Comprehensive review of state-of-the-art LLMs
  12. LLMXlorer: Large Language Model Explorer
  13. Insights from LLMXplorer
  14. Applied and Technology Implications for LLMs
  15. Social and Ethical Implications
  16. ...and 18 more sections

Figures (13)

  • Figure 1: Timeline summarising the key product developments in conversational systems
  • Figure 2: Illustration of LLM's prediction based on previous inputs
  • Figure 3: Three-stage training process of Large Language Models (LLMs): Starting from an expansive pretraining on diverse data sources and utilizing the transformer architecture, transitioning into supervised fine-tuning with labeled datasets tailored for specific tasks, and culminating in dialogue optimization to refine AI-user interactions.
  • Figure 4: Comparative analysis of annual releases: Open-source (n=89) vs. Closed-source (n=64) LLMs.
  • Figure 5: Timeline of Open Source LLMs (n=89). The x-axis displays year-month, while the y-axis shows vertical stacks depicting the total number of models (in each stack) released within one month.
  • ...and 8 more figures