Privacy-protected Retrieval-Augmented Generation for Knowledge Graph Question Answering
Yunfeng Ning, Mayi Xu, Jintao Wen, Qiankun Pi, Yuanyuan Zhu, Ming Zhong, Jiawei Jiang, Tieyun Qian
TL;DR
This work tackles privacy in knowledge-graph question answering by anonymizing KG entities with MIDs and introducing ARoG, a retrieval-augmented framework built on two abstraction strategies: relation-centric and structure-oriented. The RA and SA components transform anonymous entities and questions into semantically meaningful abstractions, enabling effective retrieval and reasoning without exposing private KG content. An abstraction-driven retrieval module and a generator produce accurate answers while preserving privacy, achieving state-of-the-art performance on WebQSP, CWQ, and GrailQA under privacy-protected settings. The results demonstrate strong privacy robustness and efficiency, highlighting ARoG’s practical potential for privacy-sensitive KGQA deployments.
Abstract
LLMs often suffer from hallucinations and outdated or incomplete knowledge. RAG is proposed to address these issues by integrating external knowledge like that in KGs into LLMs. However, leveraging private KGs in RAG systems poses significant privacy risks due to the black-box nature of LLMs and potential insecure data transmission, especially when using third-party LLM APIs lacking transparency and control. In this paper, we investigate the privacy-protected RAG scenario for the first time, where entities in KGs are anonymous for LLMs, thus preventing them from accessing entity semantics. Due to the loss of semantics of entities, previous RAG systems cannot retrieve question-relevant knowledge from KGs by matching questions with the meaningless identifiers of anonymous entities. To realize an effective RAG system in this scenario, two key challenges must be addressed: (1) How can anonymous entities be converted into retrievable information. (2) How to retrieve question-relevant anonymous entities. Hence, we propose a novel ARoG framework including relation-centric abstraction and structure-oriented abstraction strategies. For challenge (1), the first strategy abstracts entities into high-level concepts by dynamically capturing the semantics of their adjacent relations. It supplements meaningful semantics which can further support the retrieval process. For challenge (2), the second strategy transforms unstructured natural language questions into structured abstract concept paths. These paths can be more effectively aligned with the abstracted concepts in KGs, thereby improving retrieval performance. To guide LLMs to effectively retrieve knowledge from KGs, the two strategies strictly protect privacy from being exposed to LLMs. Experiments on three datasets demonstrate that ARoG achieves strong performance and privacy-robustness.