LLM-Guided Dynamic-UMAP for Personalized Federated Graph Learning

TL;DR

LG-DUMAP presents a novel framework for personalized federated graph learning that leverages large language models to provide semantic priors and few-shot signals. It integrates a Dynamic-UMAP manifold with a variational marker aggregation and cross-modal alignment to connect graph structure with LLM latent space, while enforcing privacy via secure aggregation and a moments accountant. The approach yields state-of-the-art performance in low-resource settings across node classification and link prediction tasks, with robust few-shot and cold-start capabilities and quantified privacy implications. This work offers a practical pathway to combining language priors, geometric embeddings, and privacy-preserving federation for knowledge graphs and related systems.

Abstract

We propose a method that uses large language models to assist graph machine learning under personalization and privacy constraints. The approach combines data augmentation for sparse graphs, prompt and instruction tuning to adapt foundation models to graph tasks, and in-context learning to supply few-shot graph reasoning signals. These signals parameterize a Dynamic UMAP manifold of client-specific graph embeddings inside a Bayesian variational objective for personalized federated learning. The method supports node classification and link prediction in low-resource settings and aligns language model latent representations with graph structure via a cross-modal regularizer. We outline a convergence argument for the variational aggregation procedure, describe a differential privacy threat model based on a moments accountant, and present applications to knowledge graph completion, recommendation-style link prediction, and citation and product graphs. We also discuss evaluation considerations for benchmarking LLM-assisted graph machine learning.