FewFedPIT: Towards Privacy-preserving and Few-shot Federated Instruction Tuning

TL;DR

FewFedPIT tackles privacy and data-scarcity in federated instruction tuning by combining on-device synthetic data generation with parameter isolation training and privacy-preserving local aggregation. The method leverages a federated LLM as a data generator, filters synthetic examples with an LLM-based judge using an instruction-following score, and blends public and private updates via a tunable parameter $eta$ to mitigate data leakage. Empirical results across three instruction datasets show FewFedPIT outperforming standard FedIT baselines and approaching centralized performance, while offering flexible privacy-utility tradeoffs. The approach demonstrates robust performance under non-IID conditions and provides actionable strategies to defend against training data extraction attacks in federated LLM settings.

Abstract

Instruction tuning has been identified as a crucial technique for optimizing the performance of large language models (LLMs) in generating human-aligned responses. Nonetheless, gathering diversified and superior-quality instruction data for such tuning presents notable obstacles, especially in domains with rigid privacy provisions. Federated instruction tuning (FedIT) has emerged as a promising solution, by consolidating collaborative training across multiple data owners, thereby resulting in a privacy-preserving learning model. However, FedIT encounters limitations such as scarcity of instructional data and risk of exposure to training data extraction attacks. In this paper, we propose a novel federated algorithm, FewFedPIT, designed to simultaneously enhance privacy protection and model performance of federated few-shot learning. FewFedPITcomprises three vital components on the client side: (1) synthetic data generation, which utilizes LLMs' in-context learning capacity to generate synthetic data autonomously, thus expanding the local database; (2) parameter isolation training, which individually updates the public parameters in the synthetic data and the private parameters in the local data, consequently mitigating the noise impact of the synthetic data; (3) local aggregation sharing, which mixes public and private parameters before uploading, effectively preventing data extraction attacks. Extensive experiments on three open-source datasets demonstrate the effectiveness of FewFedPITin, enhancing privacy preservation and improving federated few-shot performance.

Figures (5)

• Figure 1: (a) shows privacy data leakage of FewFedPIT with different $\beta$ and baselines proceeds with the training process. (b) shows the trade-off between privacy data leakage (y-axis) and model utility (x-axis). We measure privacy data leakage using Rouge-L, where higher values indicate more significant privacy risks. The model utility uses the WT score, representing the win and tie ratio sum. The parameter $\beta$ in FewFedPIT determines how much client privacy parameters are exposed.
• Figure 2: (a) The WT scores of various replaced synthetic data during the federated training process. (b) The contribution of FL to synthetic data generation.
• Figure 3: The comparison of different synthetic data selection methods on MedInstruct (a) and MedAlpaca (b). Our IFS method more effectively screens higher-quality samples than the other methods.
• Figure 4: The prompts used in our synthetic data generation. (a) Prompt used for generating new instructions. We randomly sample eight instructions from local data for in-context demonstration. The model is allowed to generate instructions for the new instruction. (b) Prompts used for input and output generation given the new instruction. We prompt the model with four examples followed by the new instruction. Half of the demonstration samples are selected with inputs, and the other half are selected without inputs and placed alternately.
• Figure 5: GPT4-as-a-Judge prompt for evaluating the outputs of FewFedPIT and baseline methods.