Synthesize, Partition, then Adapt: Eliciting Diverse Samples from Foundation Models

TL;DR

A novel framework that leverages the abundant synthetic data available in many domains to elicit diverse responses from foundation models, Synthesize-Partition-Adapt (SPA), that partitions data into subsets, each targeting unique aspects of the data, and trains multiple model adaptations optimized for these subsets.

Abstract

Presenting users with diverse responses from foundation models is crucial for enhancing user experience and accommodating varying preferences. However, generating multiple high-quality and diverse responses without sacrificing accuracy remains a challenge, especially when using greedy sampling. In this work, we propose a novel framework, Synthesize-Partition-Adapt (SPA), that leverages the abundant synthetic data available in many domains to elicit diverse responses from foundation models. By leveraging signal provided by data attribution methods such as influence functions, SPA partitions data into subsets, each targeting unique aspects of the data, and trains multiple model adaptations optimized for these subsets. Experimental results demonstrate the effectiveness of our approach in diversifying foundation model responses while maintaining high quality, showcased through the HumanEval and MBPP tasks in the code generation domain and several tasks in the natural language understanding domain, highlighting its potential to enrich user experience across various applications.

Figures (6)

• Figure 1: A user is expecting two diverse templates from the foundation model.
• Figure 2: pass@$1$ on HumanEval after fine-tuning on some percentage of OSS-Instruct dataset wei2023magicoder using LoRA. The plot demonstrates the diminishing returns observed with increasing amounts of data used for parameter efficient fine-tuning.
• Figure 3: An illustration of the Synthesize, Partition, then Adapt (SPA) framework. SPA partitions synthetic dataset according to data attribution scores, which can be obtained using various methods such as influence function or lexical overlap. Multiple foundation model adaptations are then trained on each subset. Sampling from the collection of these model adaptations can present users with diverse responses. SPA is not limited to a specific attribution method.
• Figure 4: How sampling temperature affects pass@$1$, pass@$5$, and Diversity Score for different methods on the HumanEval benchmark. The results are averaged over 4 checkpoints.
• Figure 5: Average KL divergence and diversity score on various natural language understanding tasks. SPA with influence function consistently outperforms the lexical overlap and random adaptations, demonstrating its effectiveness in generating diverse samples across different NLU tasks.