Private PoEtry: Private In-Context Learning via Product of Experts

TL;DR

This paper addresses privacy risks in in-context learning by reframing ICL as a Product-of-Experts (PoE) ensemble, enabling per-example privacy analysis and efficient, parallelizable private inference. By clipping log-probabilities and using the exponential mechanism, the approach provides $(\varepsilon,\delta)$-DP with respect to adjacent context sets while preserving rich soft predictive information. Theoretical analysis shows convergence to the full-context distribution under a bounded-interaction assumption, and empirical results across text, math, and vision-language tasks demonstrate substantial accuracy gains (often ~30 percentage points) over prior DP-ICL methods, along with improved empirical privacy via Membership Inference Attacks. The method offers a practical privacy-utility- efficiency trade-off for privacy-sensitive, context-based AI systems, including RAG and agentic frameworks.

Abstract

In-context learning (ICL) enables Large Language Models (LLMs) to adapt to new tasks with only a small set of examples at inference time, thereby avoiding task-specific fine-tuning. However, in-context examples may contain privacy-sensitive information that should not be revealed through model outputs. Existing differential privacy (DP) approaches to ICL are either computationally expensive or rely on heuristics with limited effectiveness, including context oversampling, synthetic data generation, or unnecessary thresholding. We reformulate private ICL through the lens of a Product-of-Experts model. This gives a theoretically grounded framework, and the algorithm can be trivially parallelized. We evaluate our method across five datasets in text classification, math, and vision-language. We find that our method improves accuracy by more than 30 percentage points on average compared to prior DP-ICL methods, while maintaining strong privacy guarantees.

Figures (7)

• Figure 1: Overview: A user makes a query to the LLM either directly or via a RAG system. The LLM responds based on examples in the context. To guarantee privacy, we use a Product-of-Experts model, which calls an LLM for each example and sums the clipped log-probabilities (in contrast to previous work that uses hard predictions, RNM, or that uses subsampling, PbS). Predictions are summed before sampling a noisy response to ascertain $\varepsilon$-DP. The results in the inlet are average accuracies from Table \ref{['tab:bigtable']} with 8 context examples.
• Figure 2: Average accuracy across AGNews, DBPedia, TREC, and GSM8k datasets. Our method performs significantly better than previous work -- especially for a small number of examples, where ICL is widely used. For $J=4$ examples, the improvement in accuracy is 30% points on average. To achieve the same accuracy, RNM would need almost 4x as many in-context examples.
• Figure 3: Predictive likelihoods on a 10-way classification task (GSM8k) with Qwen3-4B. The predictions are sorted, and the mean and std. deviation among 3000 random samples are plotted.
• Figure 4: A hyperparameter sweep for $\gamma$, with Qwen3-4B and 8 in-context examples. The mean and standard error of 25 seeds are plotted. We subsequently have used $\gamma=2$ for all experiments.
• Figure 5: Evaluation setting for in-context learning with a Vision-Language Model (VLM). The names of the classes are deliberately chosen to be nonsensical words, forcing the VLM to learn from context rather than rely on knowledge from pretraining.

Theorems & Definitions (7)

• Theorem 3.1: Differential Privacy of PoEtry algorithm
• proof
• Remark 3.3: Interpretation of Assumption
• Theorem 3.4: Convergence to Full-Context Distribution under Bounded Interaction
• proof
• Remark 3.5: Rate of convergence
• Remark 3.6: Connection to our algorithm