Information Leakage of Sentence Embeddings via Generative Embedding Inversion Attacks
Antonios Tragoudaras, Theofanis Aslanidis, Emmanouil Georgios Lionis, Marina Orozco González, Panagiotis Eustratiadis
TL;DR
The paper investigates privacy risks in sentence embeddings by reproducing Generative Embedding Inversion Attack (GEIA) across multiple embedding models and extending the analysis to detect training-data leakage. It introduces a leakage-from-training-data extension that uses masked and alternative sentence variants generated by large language models and compares log-likelihoods in the attacker space to reveal whether embeddings reveal training data. Across datasets and victim encoders, GEIA consistently outperforms non-generative baselines in reconstruction quality and demonstrates measurable leakage from training data, indicating substantial privacy risks in commonly used sentence embeddings. The findings underscore the need for defenses against both inference-time leakage and training-data leakage in embedding-based NLP systems and offer a reproducible framework for evaluating embedding security defenses.
Abstract
Text data are often encoded as dense vectors, known as embeddings, which capture semantic, syntactic, contextual, and domain-specific information. These embeddings, widely adopted in various applications, inherently contain rich information that may be susceptible to leakage under certain attacks. The GEIA framework highlights vulnerabilities in sentence embeddings, demonstrating that they can reveal the original sentences they represent. In this study, we reproduce GEIA's findings across various neural sentence embedding models. Additionally, we contribute new analysis to examine whether these models leak sensitive information from their training datasets. We propose a simple yet effective method without any modification to the attacker's architecture proposed in GEIA. The key idea is to examine differences between log-likelihood for masked and original variants of data that sentence embedding models have been pre-trained on, calculated on the embedding space of the attacker. Our findings indicate that following our approach, an adversary party can recover meaningful sensitive information related to the pre-training knowledge of the popular models used for creating sentence embeddings, seriously undermining their security. Our code is available on: https://github.com/taslanidis/GEIA