FictionalQA: A Dataset for Studying Memorization and Knowledge Acquisition

TL;DR

This work proposes a new dataset to specifically empower researchers to study the dual processes of fact memorization and verbatim sequence memorization, and conducts training experiments showing how synthetic data about fictional events can be useful for studying different forms of memorization.

Abstract

When language models are trained on textual data, they acquire both knowledge about the structure of language as well as knowledge of facts about the world. At inference time, their knowledge of facts can be leveraged to solve interesting problems and perform useful knowledge work for users. It is well known that language models can verbatim memorize long sequences from their training data. However, it is much less well understood how language models memorize facts seen during training. In this work, we propose a new dataset to specifically empower researchers to study the dual processes of fact memorization and verbatim sequence memorization. The dataset consists of synthetically-generated, webtext-like documents about fictional events, as well as question-answer pairs about the events. We conduct training experiments showing how synthetic data about fictional events can be useful for studying different forms of memorization. We also document some challenges in effectively building realistic, fictional synthetic data.

Figures (11)

• Figure 1: An illustration of the hierarchical structure of our fictional dataset. Diagram indicates how seed events are used to generate fictional documents in various styles and how questions are derived from those documents. Small liberties taken in cropping and whitespace of the example texts for visualization purposes.
• Figure 2: Samples seen as a function of optimization steps (left) and epochs completed as a function of optimization steps (right) across different splits of the fictional data. Split criteria that result in smaller training sets (primarily the Fictsheets) epoch faster because the relative batch composition is fixed at 5% fiction to 95% base webtext, regardless of the split.
• Figure 3: (Left) Loss on samples in the training and validation sets as a function of optimization step. (Right) Exact Match rate when prompting the model to generate the last 50 tokens of of the fictional document as a function of the number of epochs on all training documents in the Doc Split fictional dataset.
• Figure 4: Loss on samples in the training and validation sets of the Doc Split (left) and Event Split (right) as a function of optimization step. Legend positioning reflects the fact that these y-axes are meant to be compared, contrasting with the style of all other figure pairs.
• Figure 5: (Left) Loss on samples in the training and validation sets of the Fictsheets split as a function of optimization step. (Right) Loss on held out samples from the base webtext distribution as a function of optimizer step while training on the Doc Split (eg. the left of \ref{['fig:train-val-transfer-loss-models']}).