Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler
Zixuan Hu, Li Shen, Zhenyi Wang, Yongxian Wei, Dacheng Tao
TL;DR
Harmful fine-tuning poses safety risks to fine-tuning-as-a-service, and existing defenses rely on attack simulations that struggle to anticipate unknown threats. The authors propose Bayesian Data Scheduler (BDS), a simulation-free approach that treats data curation as Bayesian inference, learning a posterior over datapoint safety weights conditioned on the observed fine-tuning and alignment data, and applying these weights during training to mitigate harmful data influence. They present two implementations: a Bayesian Scalar Scheduler and an Amortized Bayesian Neural Scheduler, the latter enabling transfer to new data without retraining. Through extensive experiments across five downstream tasks and multiple LLM architectures, BDS achieves state-of-the-art performance, including substantial reductions in harmfulness and consistent finetune accuracy, and demonstrates robustness to OOD/ISA attacks and scalability with dataset size and alignment data. The work provides a practical, adaptive defense for real-world fine-tuning services with broad implications for safe and reliable customization of large language models.
Abstract
Harmful fine-tuning poses critical safety risks to fine-tuning-as-a-service for large language models. Existing defense strategies preemptively build robustness via attack simulation but suffer from fundamental limitations: (i) the infeasibility of extending attack simulations beyond bounded threat models due to the inherent difficulty of anticipating unknown attacks, and (ii) limited adaptability to varying attack settings, as simulation fails to capture their variability and complexity. To address these challenges, we propose Bayesian Data Scheduler (BDS), an adaptive tuning-stage defense strategy with no need for attack simulation. BDS formulates harmful fine-tuning defense as a Bayesian inference problem, learning the posterior distribution of each data point's safety attribute, conditioned on the fine-tuning and alignment datasets. The fine-tuning process is then constrained by weighting data with their safety attributes sampled from the posterior, thus mitigating the influence of harmful data. By leveraging the post hoc nature of Bayesian inference, the posterior is conditioned on the fine-tuning dataset, enabling BDS to tailor its defense to the specific dataset, thereby achieving adaptive defense. Furthermore, we introduce a neural scheduler based on amortized Bayesian learning, enabling efficient transfer to new data without retraining. Comprehensive results across diverse attack and defense settings demonstrate the state-of-the-art performance of our approach. Code is available at https://github.com/Egg-Hu/Bayesian-Data-Scheduler.