Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MeMemo: On-device Retrieval Augmentation for Private and Personalized Text Generation

Zijie J. Wang, Duen Horng Chau

TL;DR

MeMemo tackles the privacy and practicality gap in retrieval-augmented generation by moving dense retrieval onto the client. It introduces a JavaScript toolkit that adapts HNSW to browser environments, leveraging IndexedDB and Web Workers to store and search large vector databases entirely in-browser. The work provides a concrete prototype ecosystem with RAG Playground and extensive open-source tooling, documentation, and tutorials to facilitate adoption. The approach offers privacy-preserving, personalized text generation in client devices and lowers the barrier for researchers and developers to experiment with in-browser RAG workflows.

Abstract

Retrieval-augmented text generation (RAG) addresses the common limitations of large language models (LLMs), such as hallucination, by retrieving information from an updatable external knowledge base. However, existing approaches often require dedicated backend servers for data storage and retrieval, thereby limiting their applicability in use cases that require strict data privacy, such as personal finance, education, and medicine. To address the pressing need for client-side dense retrieval, we introduce MeMemo, the first open-source JavaScript toolkit that adapts the state-of-the-art approximate nearest neighbor search technique HNSW to browser environments. Developed with modern and native Web technologies, such as IndexedDB and Web Workers, our toolkit leverages client-side hardware capabilities to enable researchers and developers to efficiently search through millions of high-dimensional vectors in the browser. MeMemo enables exciting new design and research opportunities, such as private and personalized content creation and interactive prototyping, as demonstrated in our example application RAG Playground. Reflecting on our work, we discuss the opportunities and challenges for on-device dense retrieval. MeMemo is available at https://github.com/poloclub/mememo.

MeMemo: On-device Retrieval Augmentation for Private and Personalized Text Generation

TL;DR

MeMemo tackles the privacy and practicality gap in retrieval-augmented generation by moving dense retrieval onto the client. It introduces a JavaScript toolkit that adapts HNSW to browser environments, leveraging IndexedDB and Web Workers to store and search large vector databases entirely in-browser. The work provides a concrete prototype ecosystem with RAG Playground and extensive open-source tooling, documentation, and tutorials to facilitate adoption. The approach offers privacy-preserving, personalized text generation in client devices and lowers the barrier for researchers and developers to experiment with in-browser RAG workflows.

Abstract

Retrieval-augmented text generation (RAG) addresses the common limitations of large language models (LLMs), such as hallucination, by retrieving information from an updatable external knowledge base. However, existing approaches often require dedicated backend servers for data storage and retrieval, thereby limiting their applicability in use cases that require strict data privacy, such as personal finance, education, and medicine. To address the pressing need for client-side dense retrieval, we introduce MeMemo, the first open-source JavaScript toolkit that adapts the state-of-the-art approximate nearest neighbor search technique HNSW to browser environments. Developed with modern and native Web technologies, such as IndexedDB and Web Workers, our toolkit leverages client-side hardware capabilities to enable researchers and developers to efficiently search through millions of high-dimensional vectors in the browser. MeMemo enables exciting new design and research opportunities, such as private and personalized content creation and interactive prototyping, as demonstrated in our example application RAG Playground. Reflecting on our work, we discuss the opportunities and challenges for on-device dense retrieval. MeMemo is available at https://github.com/poloclub/mememo.
Paper Structure (10 sections, 1 figure)

This paper contains 10 sections, 1 figure.

Table of Contents

  1. Introduction
  2. MeMemo in Action
  3. Developing In-browser RAG Tools
  4. Prototyping with RAG Playground
  5. MeMemo Design and Implementation
  6. Adapting HNSW
  7. Optimizing for the Browsers
  8. Open-source and Easy to Use
  9. Related Work
  10. Discussion and Future Work

Figures (1)

  • Figure :