FedCoT: Federated Chain-of-Thought Distillation for Large Language Models

TL;DR

FedCoT tackles the challenge of deploying high-performing LLM reasoning in privacy-sensitive, resource-constrained environments by federating Chain-of-Thought distillation to task-specific SLMs. It introduces two privacy-preserving strategies (Exponential Mechanism and Adaptive Exponential Mechanism) to perturb prompts and rationales, enabling secure CoT knowledge transfer via a multi-task learning framework. Empirical results across multiple datasets and SLM architectures show consistent performance gains, with privacy budgets offering a tunable trade-off between utility and privacy. The approach enables practical, privacy-conscious deployment of LLM-informed SLMs for real-time text generation tasks in distributed settings, with code available in the FATE project.

Abstract

Large Language Models (LLMs) have emerged as a transformative force in artificial intelligence, demonstrating exceptional proficiency across various tasks. However, their deployment in resource-constrained environments and concerns over user data privacy pose significant challenges. In contrast, Small Language Models (SLMs) offer computational efficiency but often lag in performance. To address these issues, we propose FedCoT, a federated framework designed for the Chain-of-Thought (CoT) distillation of knowledge from LLMs to SLMs, while ensuring the preservation of clients' data privacy. FedCoT ensures secure and efficient knowledge transfer from an LLM on a high-powered server to an SLM on a resource-constrained client, while adhering to privacy requirements. Leveraging perturbed prompts and rationales generated through the CoT approach, the framework enhances the performance of the client's SLM without compromising user data privacy within a multi-task learning framework. We propose two privacy protection strategies: the Exponential Mechanism Strategy and the Adaptive Exponential Mechanism Strategy, which balance user prompt privacy and the usability of rationales. Empirical evaluation on various text generation tasks demonstrates the effectiveness of FedCoT in training task-specific SLMs with enhanced performance while prioritizing data privacy protection. Our code has been contributed to the FATE open-source project and is now publicly accessible at \textit{https://github.com/FederatedAI/FATE-LLM/tree/main/python/fate_llm/algo/fedcot}

Figures (1)

• Figure 1: The overview of our proposed FedCoT. The FedCoT comprises four key components: (1) The Prompt Encoder, which perturbs user prompts to ensure privacy; (2) The LLM, generating perturbed rationales based on the perturbed prompts; (3) The Perturbed Rationales Decoder, which decodes the perturbed rationales back into a usable form; (4) The Task-Specific SLM Enhancing via CoT Knowledge Distillation, utilizing both original labeled data and filtered rationales data for multi-task learning.