Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Large Language Models Meet Vector Databases: A Survey

Zhi Jing, Yongye Su, Yikun Han, Bo Yuan, Haiyun Xu, Chunjiang Liu, Kehai Chen, Min Zhang

TL;DR

This survey analyzes how Vector Databases (VecDBs) can address core limitations of Large Language Models (LLMs), including hallucinations, outdated knowledge, high costs, and memory constraints. It surveys VecDB fundamentals, retrieval-augmented generation (RAG), semantic caching, and memory-enabled architectures, highlighting multimodal extensions and retrieval optimizations. The paper discusses practical challenges such as hybrid search needs, multi-modality, data preprocessing, multi-tenancy, and knowledge conflicts, offering a roadmap for future work and enabling more capable, scalable AI systems. Overall, it articulates a coherent framework for integrating VecDBs with LLMs to enhance data handling, knowledge extraction, and real-time adaptability in production settings.

Abstract

This survey explores the synergistic potential of Large Language Models (LLMs) and Vector Databases (VecDBs), a burgeoning but rapidly evolving research area. With the proliferation of LLMs comes a host of challenges, including hallucinations, outdated knowledge, prohibitive commercial application costs, and memory issues. VecDBs emerge as a compelling solution to these issues by offering an efficient means to store, retrieve, and manage the high-dimensional vector representations intrinsic to LLM operations. Through this nuanced review, we delineate the foundational principles of LLMs and VecDBs and critically analyze their integration's impact on enhancing LLM functionalities. This discourse extends into a discussion on the speculative future developments in this domain, aiming to catalyze further research into optimizing the confluence of LLMs and VecDBs for advanced data handling and knowledge extraction capabilities.

When Large Language Models Meet Vector Databases: A Survey

TL;DR

This survey analyzes how Vector Databases (VecDBs) can address core limitations of Large Language Models (LLMs), including hallucinations, outdated knowledge, high costs, and memory constraints. It surveys VecDB fundamentals, retrieval-augmented generation (RAG), semantic caching, and memory-enabled architectures, highlighting multimodal extensions and retrieval optimizations. The paper discusses practical challenges such as hybrid search needs, multi-modality, data preprocessing, multi-tenancy, and knowledge conflicts, offering a roadmap for future work and enabling more capable, scalable AI systems. Overall, it articulates a coherent framework for integrating VecDBs with LLMs to enhance data handling, knowledge extraction, and real-time adaptability in production settings.

Abstract

This survey explores the synergistic potential of Large Language Models (LLMs) and Vector Databases (VecDBs), a burgeoning but rapidly evolving research area. With the proliferation of LLMs comes a host of challenges, including hallucinations, outdated knowledge, prohibitive commercial application costs, and memory issues. VecDBs emerge as a compelling solution to these issues by offering an efficient means to store, retrieve, and manage the high-dimensional vector representations intrinsic to LLM operations. Through this nuanced review, we delineate the foundational principles of LLMs and VecDBs and critically analyze their integration's impact on enhancing LLM functionalities. This discourse extends into a discussion on the speculative future developments in this domain, aiming to catalyze further research into optimizing the confluence of LLMs and VecDBs for advanced data handling and knowledge extraction capabilities.
Paper Structure (30 sections, 2 figures, 1 table)

This paper contains 30 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Large Language Models
  4. Development of Language Models
  5. Power of Pre-training
  6. Are We There Yet? Challenges Faced by Pure LLMs
  7. Vector Databases, the V-factor
  8. A distinct database optimized for vector data
  9. Efficient vector retrieval within VDBMS
  10. A feasible solution of knowledge base for LLMs
  11. Synergizing LLMs with VecDBs: Enhancements and Innovations
  12. VecDB as an External Knowledge Base: Retrieval-Augmented Generation (RAG)
  13. Development of RAG Paradigm
  14. RAG with VecDBs
  15. VecDB as a Cost-effective Semantic Cache
  16. ...and 15 more sections

Figures (2)

  • Figure 1: A sample RAG framework that uses VecDBs.
  • Figure 2: An overview of semantic cache for GPTs that utilizes VecDBs.