RemoteRAG: A Privacy-Preserving LLM Cloud RAG Service

TL;DR

The paper addresses privacy leakage in cloud-based retrieval-augmented generation (RAG) services where user queries may reveal sensitive information. It introduces RemoteRAG, a privacy-preserving cloud RAG framework built on $(n,\epsilon)$-DistanceDP that perturbs the user embedding with a radius drawn from $\mathrm{Gamma}(n, 1/\epsilon)$ and limits the search to a small, privacy-bounded set of documents. The approach combines two modules: (i) a perturbation-based privacy mechanism and (ii) cryptographic retrieval using partially homomorphic encryption for cosine distance and a $k$-out-of-$k'$ oblivious transfer to safely obtain the top-$k$ indices. Theoretical analysis guarantees that the top-$k$ original documents are contained within the retrieved $k'$ set under the specified privacy budget, while experimental results demonstrate resistance to embedding-inversion attacks and near-lossless retrieval with low latency (≈$0.67$ s and $46.66$ KB transmission for a $10^6$-document corpus). Overall, RemoteRAG presents a practical framework for privacy-aware cloud RAG with favorable privacy, efficiency, and accuracy characteristics compared to privacy-ignorant and privacy-conscious baselines.

Abstract

Retrieval-augmented generation (RAG) improves the service quality of large language models by retrieving relevant documents from credible literature and integrating them into the context of the user query. Recently, the rise of the cloud RAG service has made it possible for users to query relevant documents conveniently. However, directly sending queries to the cloud brings potential privacy leakage. In this paper, we are the first to formally define the privacy-preserving cloud RAG service to protect the user query and propose RemoteRAG as a solution regarding privacy, efficiency, and accuracy. For privacy, we introduce $(n,ε)$-DistanceDP to characterize privacy leakage of the user query and the leakage inferred from relevant documents. For efficiency, we limit the search range from the total documents to a small number of selected documents related to a perturbed embedding generated from $(n,ε)$-DistanceDP, so that computation and communication costs required for privacy protection significantly decrease. For accuracy, we ensure that the small range includes target documents related to the user query with detailed theoretical analysis. Experimental results also demonstrate that RemoteRAG can resist existing embedding inversion attack methods while achieving no loss in retrieval under various settings. Moreover, RemoteRAG is efficient, incurring only $0.67$ seconds and $46.66$KB of data transmission ($2.72$ hours and $1.43$ GB with the non-optimized privacy-preserving scheme) when retrieving from a total of $10^6$ documents.

Figures (15)

• Figure 1: The flowchart of RemoteRAG. Module 1 preserves privacy with $(n,\epsilon)$-DistanceDP and improves efficiency by limiting the search range. Module 2 retrieves documents with different choices based on leakage circumstances.
• Figure 2: The probability density function of different gamma distributions within $[0.06,0.14]$ range. ($\epsilon=10n$)
• Figure 3: Oblique projection. Top $k'$ documents related to $e_{k'}$ include top $k$ documents related to $e_k$.
• Figure 4: Orthographic projection. $\alpha_{k'}=\alpha_k+\Delta\alpha_k$.
• Figure 6: SacreBLEU-$r$

Theorems & Definitions (16)

• Definition 1: $(n,\epsilon)$-DistanceDP
• Lemma 1
• Theorem 1
• Definition 2: Distance metrics
• Theorem 2
• Theorem 3
• Lemma 2: Repeated from \ref{['lem:k_alpha_k']}
• proof
• Theorem 4: Repeated from \ref{['thm:kpr_k']}
• proof