Self-QA: Unsupervised Knowledge Guided Language Model Alignment

TL;DR

Problem: Creating supervised instruction-tuning data is costly and hard to scale. Approach: Self-QA generates instruction data from unsupervised knowledge via knowledge-guided instruction generation and machine reading comprehension, with a filtering step. Contributions: introduces the Self-QA framework and demonstrates diverse, domain-specific, correct instruction data across unsupervised corpora. Significance: enables scalable, low-annotation instruction-tuning and broader applicability of instruction-following models.

Abstract

Large-scale language models like ChatGPT and GPT-4 have gained attention for their impressive conversational and generative capabilities. However, the creation of supervised paired question-answering data for instruction tuning presents formidable challenges. This endeavor necessitates substantial human effort for data annotation and wrestles with issues concerning data quality, diversity, accuracy, and other related factors. To overcome these obstacles, we introduce an innovative framework named Self-QA, which replaces the traditional practice of human-written instruction seeds with a vast amount of unsupervised knowledge, enabling the model to generate a larger quantity of correct and domain-specific instruction data. The effectiveness of our proposed method is demonstrated through experiments conducted on unsupervised corpora from various domains.

Figures (2)

• Figure 1: The pipeline of Self-QA.
• Figure 2: Examples of transformation of unsupervised structured data.