PrivGemo: Privacy-Preserving Dual-Tower Graph Retrieval for Empowering LLM Reasoning with Memory Augmentation

TL;DR

PrivGemo tackles privacy-preserving KG-grounded reasoning for LLMs by decoupling local KG grounding from remote reasoning through a dual-LLM controller and a privacy-aware memory system. It constructs a local question-specific raw subgraph and an anonymized view for remote analysis, enabling coherent multi-hop reasoning while limiting exposure of sensitive graph structure and content. A memory-augmented hierarchical reasoning process gates remote calls, guides exploration, and reuses verified reasoning patterns, achieving state-of-the-art results on six KGQA benchmarks and enabling smaller models to match stronger baselines under privacy constraints. The approach provides practical privacy guarantees for enterprise KG data with robust performance and adaptability across diverse KGQA tasks.

Abstract

Knowledge graphs (KGs) provide structured evidence that can ground large language model (LLM) reasoning for knowledge-intensive question answering. However, many practical KGs are private, and sending retrieved triples or exploration traces to closed-source LLM APIs introduces leakage risk. Existing privacy treatments focus on masking entity names, but they still face four limitations: structural leakage under semantic masking, uncontrollable remote interaction, fragile multi-hop and multi-entity reasoning, and limited experience reuse for stability and efficiency. To address these issues, we propose PrivGemo, a privacy-preserving retrieval-augmented framework for KG-grounded reasoning with memory-guided exposure control. PrivGemo uses a dual-tower design to keep raw KG knowledge local while enabling remote reasoning over an anonymized view that goes beyond name masking to limit both semantic and structural exposure. PrivGemo supports multi-hop, multi-entity reasoning by retrieving anonymized long-hop paths that connect all topic entities, while keeping grounding and verification on the local KG. A hierarchical controller and a privacy-aware experience memory further reduce unnecessary exploration and remote interactions. Comprehensive experiments on six benchmarks show that PrivGemo achieves overall state-of-the-art results, outperforming the strongest baseline by up to 17.1%. Furthermore, PrivGemo enables smaller models (e.g., Qwen3-4B) to achieve reasoning performance comparable to that of GPT-4-Turbo.

Figures (10)

• Figure 1: Representative workflow of three LLM reasoning paradigms.
• Figure 2: Performance vs. Anonymization Ratio
• Figure 3: Illustration of Memory-Gated Brain and dual-LLM question analysis.
• Figure 4: Overview of the PrivGemo framework with privacy-aware memory and dual-LLM hierarchical reasoning. Privacy-aware initialization constructs a question-specific (raw and anonymized) KG view after topic entity recognition and subgraph detection. Privacy-aware memory retrieves and updates experience records that store prior search depth, exploration mode, question analysis, and trajectories. Dual-LLM hierarchical reasoning uses the retrieved (or newly generated) indicator and split questions to run multi-stage exploration and build a question tree with forward and backtracking steps. Evidence retrieval and pruning integrates experience-guided fuzzy selection, brain-assisted path selection, and sufficiency checks to iteratively prune evidence and decide when to stop and answer.
• Figure 5: The lengths of the ground-truth SPARQL queries within the CWQ and WebQSP datasets.

Theorems & Definitions (2)

• Definition 1: Reasoning Path
• Definition 2: Entity Path