Tuning without Peeking: Provable Privacy and Generalization Bounds for LLM Post-Training
Ismail Labiad, Mathurin Videau, Matthieu Kowalski, Marc Schoenauer, Alessandro Leite, Julia Kempe, Olivier Teytaud
TL;DR
BBoxER addresses privacy and robustness in LLM post-training by leveraging comparison-based black-box optimization to learn small data-derived updates without gradients. It formalizes an information bottleneck through a compression of the optimization trace, and derives non-vacuous generalization bounds along with provable privacy and poisoning guarantees. Empirically, a few hundred BBoxER evaluations yield consistent improvements on reasoning benchmarks for billion-scale LLMs while resisting membership inference and data-extraction attacks. The approach serves as a safe, modular add-on atop gradient-based training, enabling private, scalable post-training in privacy-sensitive or restricted settings.
Abstract
Gradient-based optimization is the workhorse of deep learning, offering efficient and scalable training via backpropagation. However, exposing gradients during training can leak sensitive information about the underlying data, raising privacy and security concerns such as susceptibility to data poisoning attacks. In contrast, black box optimization methods, which treat the model as an opaque function, relying solely on function evaluations to guide optimization, offer a promising alternative in scenarios where data access is restricted, adversarial risks are high, or overfitting is a concern. This paper introduces BBoxER, an evolutionary black-box method for LLM post-training that induces an information bottleneck via implicit compression of the training data. Leveraging the tractability of information flow, we provide non-vacuous generalization bounds and strong theoretical guarantees for differential privacy, robustness to data poisoning attacks, and extraction attacks. In experiments with LLMs, we demonstrate empirically that black-box optimization methods-despite the scalability and computational challenges inherent to black-box approaches-are able to learn, showing how a few iterations of BBoxER improve performance, generalize well on a benchmark of reasoning datasets, and are robust to membership inference attacks. This positions BBoxER as an attractive add-on on top of gradient-based optimization, offering suitability for deployment in restricted or privacy-sensitive environments while also providing non-vacuous generalization guarantees.