Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Zero2Text: Zero-Training Cross-Domain Inversion Attacks on Textual Embeddings

Doohyun Kim, Donghwa Kang, Kyungjae Lee, Hyeongboo Baek, Brent Byunghoon Kang

TL;DR

Zero2Text presents a training‑free, cross‑domain inversion attack that recovers text from unknown textual embeddings by coupling LLM‑driven generation with an online, instance‑level alignment learned via ridge regression. It bypasses offline data requirements and domain restrictions, enabling robust inversion even under strict black‑box and cross‑domain scenarios. The method achieves state‑of‑the‑art fidelity on MS MARCO against OpenAI’s victim model, notably yielding approx. 1.8× higher ROUGE‑L and 6.4× higher BLEU‑2 scores than prior baselines, while incurring minimal API query cost. Defense experiments show standard techniques like differential privacy offer limited protection against such adaptive, online attacks, underscoring urgent privacy‑preserving needs for embedding APIs and vector databases.

Abstract

The proliferation of retrieval-augmented generation (RAG) has established vector databases as critical infrastructure, yet they introduce severe privacy risks via embedding inversion attacks. Existing paradigms face a fundamental trade-off: optimization-based methods require computationally prohibitive queries, while alignment-based approaches hinge on the unrealistic assumption of accessible in-domain training data. These constraints render them ineffective in strict black-box and cross-domain settings. To dismantle these barriers, we introduce Zero2Text, a novel training-free framework based on recursive online alignment. Unlike methods relying on static datasets, Zero2Text synergizes LLM priors with a dynamic ridge regression mechanism to iteratively align generation to the target embedding on-the-fly. We further demonstrate that standard defenses, such as differential privacy, fail to effectively mitigate this adaptive threat. Extensive experiments across diverse benchmarks validate Zero2Text; notably, on MS MARCO against the OpenAI victim model, it achieves 1.8x higher ROUGE-L and 6.4x higher BLEU-2 scores compared to baselines, recovering sentences from unknown domains without a single leaked data pair.

Zero2Text: Zero-Training Cross-Domain Inversion Attacks on Textual Embeddings

TL;DR

Zero2Text presents a training‑free, cross‑domain inversion attack that recovers text from unknown textual embeddings by coupling LLM‑driven generation with an online, instance‑level alignment learned via ridge regression. It bypasses offline data requirements and domain restrictions, enabling robust inversion even under strict black‑box and cross‑domain scenarios. The method achieves state‑of‑the‑art fidelity on MS MARCO against OpenAI’s victim model, notably yielding approx. 1.8× higher ROUGE‑L and 6.4× higher BLEU‑2 scores than prior baselines, while incurring minimal API query cost. Defense experiments show standard techniques like differential privacy offer limited protection against such adaptive, online attacks, underscoring urgent privacy‑preserving needs for embedding APIs and vector databases.

Abstract

The proliferation of retrieval-augmented generation (RAG) has established vector databases as critical infrastructure, yet they introduce severe privacy risks via embedding inversion attacks. Existing paradigms face a fundamental trade-off: optimization-based methods require computationally prohibitive queries, while alignment-based approaches hinge on the unrealistic assumption of accessible in-domain training data. These constraints render them ineffective in strict black-box and cross-domain settings. To dismantle these barriers, we introduce Zero2Text, a novel training-free framework based on recursive online alignment. Unlike methods relying on static datasets, Zero2Text synergizes LLM priors with a dynamic ridge regression mechanism to iteratively align generation to the target embedding on-the-fly. We further demonstrate that standard defenses, such as differential privacy, fail to effectively mitigate this adaptive threat. Extensive experiments across diverse benchmarks validate Zero2Text; notably, on MS MARCO against the OpenAI victim model, it achieves 1.8x higher ROUGE-L and 6.4x higher BLEU-2 scores compared to baselines, recovering sentences from unknown domains without a single leaked data pair.
Paper Structure (32 sections, 5 equations, 4 figures, 7 tables)

This paper contains 32 sections, 5 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Inversion Attacks on Embeddings
  4. Defense Mechanisms Against Inversion Attacks
  5. Methodology
  6. Problem Formulation
  7. Design Principles
  8. Iterative Token Generation
  9. LLM-based Generation.
  10. Diversity-Aware Filtering.
  11. Online Verification and Optimization
  12. Online Projection Optimization.
  13. Confidence-Aware Scoring (Verification).
  14. Evaluation
  15. Experimental Setting
  16. ...and 17 more sections

Figures (4)

  • Figure 1: Illustration of prevailing inversion paradigms. (a) Direct inversion methods (e.g., Vec2Text) require extracting massive text-vector pairs (❶) to train a specialized decoder (❷). (b) Embedder alignment methods (e.g., ALGEN) train a decoder on general corpora (❶) and rely on few-shot leaked pairs from DB to align the victim embedder (❷). Both paradigms struggle with domain shifts when the target domain is unknown.
  • Figure 2: The recursive workflow of Zero2Text: ❶ Generate diverse candidate tokens via LLM; ❷ Project local embeddings to the target space; ❸ Group candidates by confidence; ❹ Query the victim model for the top-$(K_A\times\gamma^{t-1})$ candidates; ❺ Update the alignment matrix online; ❻ Re-project non-queried candidates with the updated matrix; ❼ Re-score all candidates; and ❽ Select the best sequences via Beam Search for the next iteration.
  • Figure 3: Comparison inversion performance between Vec2Text, ALGEN and Zero2Text depending on the number of queries.
  • Figure 4: Scalability analysis of inversion attack performance between ALGEN and Zero2Text.