"Someone Hid It": Query-Agnostic Black-Box Attacks on LLM-Based Retrieval

TL;DR

This work reveals a vulnerability of LLM-based retrieval systems to query-agnostic black-box attacks that require no access to the victim model, corpus, or queries. It develops a theoretical framework and a Transferable Query-Document Adversarial (DQ-A) learning method that injects tokens into documents via a surrogate model, achieving zero-shot transferability across multiple LLM retrievers. A key contribution is the ε-pε-Precise retriever concept, plus a min-max optimization with a word-embedding surrogate that enhances transferability. Experiments on BRIGHT datasets show meaningful retrieval degradation across most retrievers, underscoring the need for defenses that harden retrieval against such adversarial edits rather than relying solely on post-retrieval detection.

Abstract

Large language models (LLMs) have been serving as effective backbones for retrieval systems, including Retrieval-Augmentation-Generation (RAG), Dense Information Retriever (IR), and Agent Memory Retrieval. Recent studies have demonstrated that such LLM-based Retrieval (LLMR) is vulnerable to adversarial attacks, which manipulates documents by token-level injections and enables adversaries to either boost or diminish these documents in retrieval tasks. However, existing attack studies mainly (1) presume a known query is given to the attacker, and (2) highly rely on access to the victim model's parameters or interactions, which are hardly accessible in real-world scenarios, leading to limited validity. To further explore the secure risks of LLMR, we propose a practical black-box attack method that generates transferable injection tokens based on zero-shot surrogate LLMs without need of victim queries or victim models knowledge. The effectiveness of our attack raises such a robustness issue that similar effects may arise from benign or unintended document edits in the real world. To achieve our attack, we first establish a theoretical framework of LLMR and empirically verify it. Under the framework, we simulate the transferable attack as a min-max problem, and propose an adversarial learning mechanism that finds optimal adversarial tokens with learnable query samples. Our attack is validated to be effective on benchmark datasets across popular LLM retrievers.

Figures (6)

• Figure 1: In many practical scenarios, attackers may hope to hide web documents from retrieval systems. These websites usually allow normal public users to edit in format of content contribution or discussion replies.
• Figure 2: Illustration of LLM-based Retrieval. Documents in the corpus are firstly embedded in the last-hidden embeddings and stored. When a user query comes and get embedded, it matches relevant documents in embedding similarity in high efficiency.
• Figure 3: We sample 40 knowledge contexts on each of four roughly-defined topics and visualize their LLMR embeddings by Principal Component Analysis (PCA) reduction. Embeddings of contexts within the same topic (R:science, G:politic, Y:movie, B:architecture) tend to cluster together in the embedding space.
• Figure 4: The DQ-A learning pipeline of our attack method. Query samples are first generated by a third party casual LLM. Then in every learning step, injected document tokens are first optimized away from queries, and all queries tokens are optimized towards the document. Both surrogate and Casual LLMs require no learning.
• Figure 5: Impact of Different $|\mathcal{S}|$

Theorems & Definitions (2)

• Definition 3.1: $\epsilon$-$p_{\epsilon}$-Precise Retriever
• Lemma 3.2: Transferability of Attack on $g$ to $f$