Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Safe-FedLLM: Delving into the Safety of Federated Large Language Models

Mingxiang Tao, Yu Tian, Wenxuan Tu, Yue Yang, Xue Yang, Xiangyan Tang

TL;DR

This work assesses safety risks in federated fine-tuning of large language models (FedLLM) and identifies that malicious clients can undermine model safety, with LoRA weight patterns offering separable, endogenous safety signals. It introduces Safe-FedLLM, a probe-based defense combining a LoRA-Probe and three defense modules (Step-, Client-, Shadow-Level) plus a security-weighted aggregation strategy to filter malicious updates without sacrificing benign performance. The approach leverages offline-learned LoRA deltas and online monitoring to produce per-client security factors that govern aggregation, including a shadow-LoRA branch for robust detection. Across multiple backbones (e.g., Llama3.1-8B, Qwen2.5-7B) and varying malicious-client ratios, Safe-FedLLM improves safety metrics (Rule, MD-Judge, RM) with only modest overhead, demonstrating practical robustness for open FedLLM deployments.

Abstract

Federated learning (FL) addresses data privacy and silo issues in large language models (LLMs). Most prior work focuses on improving the training efficiency of federated LLMs. However, security in open environments is overlooked, particularly defenses against malicious clients. To investigate the safety of LLMs during FL, we conduct preliminary experiments to analyze potential attack surfaces and defensible characteristics from the perspective of Low-Rank Adaptation (LoRA) weights. We find two key properties of FL: 1) LLMs are vulnerable to attacks from malicious clients in FL, and 2) LoRA weights exhibit distinct behavioral patterns that can be filtered through simple classifiers. Based on these properties, we propose Safe-FedLLM, a probe-based defense framework for federated LLMs, constructing defenses across three dimensions: Step-Level, Client-Level, and Shadow-Level. The core concept of Safe-FedLLM is to perform probe-based discrimination on the LoRA weights locally trained by each client during FL, treating them as high-dimensional behavioral features and using lightweight classification models to determine whether they possess malicious attributes. Extensive experiments demonstrate that Safe-FedLLM effectively enhances the defense capability of federated LLMs without compromising performance on benign data. Notably, our method effectively suppresses malicious data impact without significant impact on training speed, and remains effective even with many malicious clients. Our code is available at: https://github.com/dmqx/Safe-FedLLM.

Safe-FedLLM: Delving into the Safety of Federated Large Language Models

TL;DR

This work assesses safety risks in federated fine-tuning of large language models (FedLLM) and identifies that malicious clients can undermine model safety, with LoRA weight patterns offering separable, endogenous safety signals. It introduces Safe-FedLLM, a probe-based defense combining a LoRA-Probe and three defense modules (Step-, Client-, Shadow-Level) plus a security-weighted aggregation strategy to filter malicious updates without sacrificing benign performance. The approach leverages offline-learned LoRA deltas and online monitoring to produce per-client security factors that govern aggregation, including a shadow-LoRA branch for robust detection. Across multiple backbones (e.g., Llama3.1-8B, Qwen2.5-7B) and varying malicious-client ratios, Safe-FedLLM improves safety metrics (Rule, MD-Judge, RM) with only modest overhead, demonstrating practical robustness for open FedLLM deployments.

Abstract

Federated learning (FL) addresses data privacy and silo issues in large language models (LLMs). Most prior work focuses on improving the training efficiency of federated LLMs. However, security in open environments is overlooked, particularly defenses against malicious clients. To investigate the safety of LLMs during FL, we conduct preliminary experiments to analyze potential attack surfaces and defensible characteristics from the perspective of Low-Rank Adaptation (LoRA) weights. We find two key properties of FL: 1) LLMs are vulnerable to attacks from malicious clients in FL, and 2) LoRA weights exhibit distinct behavioral patterns that can be filtered through simple classifiers. Based on these properties, we propose Safe-FedLLM, a probe-based defense framework for federated LLMs, constructing defenses across three dimensions: Step-Level, Client-Level, and Shadow-Level. The core concept of Safe-FedLLM is to perform probe-based discrimination on the LoRA weights locally trained by each client during FL, treating them as high-dimensional behavioral features and using lightweight classification models to determine whether they possess malicious attributes. Extensive experiments demonstrate that Safe-FedLLM effectively enhances the defense capability of federated LLMs without compromising performance on benign data. Notably, our method effectively suppresses malicious data impact without significant impact on training speed, and remains effective even with many malicious clients. Our code is available at: https://github.com/dmqx/Safe-FedLLM.
Paper Structure (35 sections, 13 equations, 4 figures, 9 tables)

This paper contains 35 sections, 13 equations, 4 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Federated Large Language Models
  4. Defenses in Federated Learning
  5. Preliminary
  6. Preliminary Experimental Setup
  7. Training Environment.
  8. Evaluation Metrics.
  9. Analysis
  10. Motivations
  11. Methods
  12. LoRA-Probe
  13. Offline Training.
  14. Malicious Feature Detection.
  15. Defense Modules
  16. ...and 20 more sections

Figures (4)

  • Figure 1: Traditional FedLLM vs. Safe-FedLLM
  • Figure 2: LDA+PCA visualization of the first-layer LoRA $B$-matrix weight distributions under four dataset combinations. The first row corresponds to Llama3.1-8B, and the second row corresponds to Qwen2.5-7B.
  • Figure 3: Overview of the Safe-FedLLM framework, which consists of LoRA-Probe and Safety Defense Module.
  • Figure 4: Precision of Llama3.1-8B under different defense modules at 30% malicious ratio on BeaverTails & LMSYS-Chat.