Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging Unstructured Text Data for Federated Instruction Tuning of Large Language Models

Rui Ye, Rui Ge, Yuchi Fengting, Jingyi Chai, Yanfeng Wang, Siheng Chen

TL;DR

A novel and flexible framework FedIT-U2S is proposed, which can automatically transform unstructured corpus into structured data for federated instruction tuning and can be applied to diverse scenarios as long as the client holds valuable text corpus, broadening the application scope of federated instruction tuning.

Abstract

Federated instruction tuning enables multiple clients to collaboratively fine-tune a shared large language model (LLM) that can follow humans' instructions without directly sharing raw data. However, existing literature impractically requires that all the clients readily hold instruction-tuning data (i.e., structured instruction-response pairs), which necessitates massive human annotations since clients' data is usually unstructured text instead. Addressing this, we propose a novel and flexible framework FedIT-U2S, which can automatically transform unstructured corpus into structured data for federated instruction tuning. FedIT-U2S consists two key steps: (1) few-shot instruction-tuning data generation, where each unstructured data piece together with several examples is combined to prompt an LLM in generating an instruction-response pair. To further enhance the flexibility, a retrieval-based example selection technique is proposed, where the examples are automatically selected based on the relatedness between the client's data piece and example pool, bypassing the need of determining examples in advance. (2) A typical federated instruction tuning process based on the generated data. Overall, FedIT-U2S can be applied to diverse scenarios as long as the client holds valuable text corpus, broadening the application scope of federated instruction tuning. We conduct a series of experiments on three domains (medicine, knowledge, and math), showing that our proposed FedIT-U2S can consistently and significantly brings improvement over the base LLM.

Leveraging Unstructured Text Data for Federated Instruction Tuning of Large Language Models

TL;DR

A novel and flexible framework FedIT-U2S is proposed, which can automatically transform unstructured corpus into structured data for federated instruction tuning and can be applied to diverse scenarios as long as the client holds valuable text corpus, broadening the application scope of federated instruction tuning.

Abstract

Federated instruction tuning enables multiple clients to collaboratively fine-tune a shared large language model (LLM) that can follow humans' instructions without directly sharing raw data. However, existing literature impractically requires that all the clients readily hold instruction-tuning data (i.e., structured instruction-response pairs), which necessitates massive human annotations since clients' data is usually unstructured text instead. Addressing this, we propose a novel and flexible framework FedIT-U2S, which can automatically transform unstructured corpus into structured data for federated instruction tuning. FedIT-U2S consists two key steps: (1) few-shot instruction-tuning data generation, where each unstructured data piece together with several examples is combined to prompt an LLM in generating an instruction-response pair. To further enhance the flexibility, a retrieval-based example selection technique is proposed, where the examples are automatically selected based on the relatedness between the client's data piece and example pool, bypassing the need of determining examples in advance. (2) A typical federated instruction tuning process based on the generated data. Overall, FedIT-U2S can be applied to diverse scenarios as long as the client holds valuable text corpus, broadening the application scope of federated instruction tuning. We conduct a series of experiments on three domains (medicine, knowledge, and math), showing that our proposed FedIT-U2S can consistently and significantly brings improvement over the base LLM.
Paper Structure (10 sections, 3 figures, 2 tables)

This paper contains 10 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Pipeline of FedIT-U2S
  5. Retrieval-based Example Selection for Few-Shot Generation
  6. Experiments
  7. Experimental Details
  8. Experimental Results
  9. Conclusions
  10. Appendix

Figures (3)

  • Figure 1: Overview of our proposed FedIT-U2S. It consists of two key steps: data generation and FedIT. Data generation is required only once before FedIT. (1) For each raw unstructured data piece, clients select a few examples by retrieving from an example database to construct a few-shot template, prompting the LLM to generate an instruction-response pair. (2) Typical federated instruction tuning starts based on the generated structured instruction-tuning data.
  • Figure 2: The t-SNE visualization of embeddings of instruction-response pairs in PubMedQA, HotpotQA and AQUA_RAT. Blue dots represent generated data, while red dots represent human-annotated data. The close proximity of each pair of red and blue dots indicates that the generated data closely aligns with the human-annotated data.
  • Figure 3: Example illustration.