Federated In-Context LLM Agent Learning
Panlong Wu, Kangshuo Li, Junbao Nan, Fangxin Wang
TL;DR
The paper tackles privacy and communication bottlenecks in federated training of LLM agents by introducing Federated In-Context LLM Agent Learning (FICAL). It shifts from transmitting bulky model parameters to exchanging privacy-preserving knowledge compendiums generated by an LLM-enhanced Knowledge Compendiums Generation (KCG) module, achieving $O(1)$ communication independent of model size. A Retrieval Augmented Generation (RAG)–based Tool Learning and Utilizing (TLU) module uses the aggregated global compendium as a teacher to improve tool usage in LLM agents, mitigating long-context issues. Empirical results show FICAL is competitive with state-of-the-art baselines while reducing communication costs by about $3.33\times10^5$, and the RAG component contributes an additional $7.6\%$ accuracy gain. This approach enables scalable, privacy-preserving, tool-aware FL for large-scale LLMs with practical potential for real-world deployments.
Abstract
Large Language Models (LLMs) have revolutionized intelligent services by enabling logical reasoning, tool use, and interaction with external systems as agents. The advancement of LLMs is frequently hindered by the scarcity of high-quality data, much of which is inherently sensitive. Federated learning (FL) offers a potential solution by facilitating the collaborative training of distributed LLMs while safeguarding private data. However, FL frameworks face significant bandwidth and computational demands, along with challenges from heterogeneous data distributions. The emerging in-context learning capability of LLMs offers a promising approach by aggregating natural language rather than bulky model parameters. Yet, this method risks privacy leakage, as it necessitates the collection and presentation of data samples from various clients during aggregation. In this paper, we propose a novel privacy-preserving Federated In-Context LLM Agent Learning (FICAL) algorithm, which to our best knowledge for the first work unleashes the power of in-context learning to train diverse LLM agents through FL. In our design, knowledge compendiums generated by a novel LLM-enhanced Knowledge Compendiums Generation (KCG) module are transmitted between clients and the server instead of model parameters in previous FL methods. Apart from that, an incredible Retrieval Augmented Generation (RAG) based Tool Learning and Utilizing (TLU) module is designed and we incorporate the aggregated global knowledge compendium as a teacher to teach LLM agents the usage of tools. We conducted extensive experiments and the results show that FICAL has competitive performance compared to other SOTA baselines with a significant communication cost decrease of $\mathbf{3.33\times10^5}$ times.