Accelerating Large Language Model Pretraining via LFR Pedagogy: Learn, Focus, and Review

TL;DR

This work tackles the high cost of large-scale autoregressive pretraining by introducing Learn-Focus-Review (LFR), a dynamic data-training paradigm that mimics spaced repetition. By tracking perplexity across data blocks and reweighting exposure toward challenging regions while periodically reviewing all data, LFR achieves lower perplexity and higher downstream accuracy using only a fraction of training tokens. Across Llama-2 and GPT-2 families, LFR matches or surpasses larger baselines with up to ~20x fewer iterations, and demonstrates clear benefits over static data-selection methods in diverse tasks. The approach reveals that LLMs initially learn conversational and anecdotal data before retaining factual and coding knowledge, highlighting the value of adaptive, memory-inspired training protocols for efficient pretraining.

Abstract

Traditional Large Language Model (LLM) pretraining relies on autoregressive language modeling with randomly sampled data from web-scale datasets. Inspired by human learning techniques like spaced repetition, we hypothesize that random sampling leads to high training costs, lower-quality models, and significant data forgetting. To address these inefficiencies, we propose the Learn-Focus-Review (LFR) paradigm -- a dynamic training approach that adapts to the model's learning progress. LFR tracks the model's learning performance across data blocks (sequences of tokens) and prioritizes revisiting challenging regions of the dataset that are more prone to being forgotten, enabling better retention and more efficient learning. Using the LFR paradigm, we pretrained Llama and GPT models on the SlimPajama and OpenWebText datasets, respectively. These models were evaluated on downstream tasks across various domains, including question answering, problem-solving, commonsense reasoning, language modeling, and translation. Compared to baseline models trained on the full datasets, LFR consistently achieved lower perplexity and higher accuracy, while using only 5%--19% of the training tokens. Furthermore, LFR matched the performance of industry-standard Pythia models with up to 2$\times$ the parameter count, using just 3.2% of the training tokens, demonstrating its effectiveness and efficiency.

Figures (10)

• Figure 1: LLM training timeline
• Figure 2: PPL trajectories of data samples from the SlimPajama dataset as processed by the Llama-300M model, focusing on a subset of 50 samples for clarity. Notably, 78.5% of the samples exhibit this behavior, characterized by multiple descent patterns rather than a steady decline. This indicates that the model frequently forgets and relearns data during training, highlighting inefficiencies in traditional training dynamics
• Figure 3: PPLs of data samples being forgotton by the GPT2-345M model. This multi-descent behavior is exhibited by 20% of the data.
• Figure 4: PPL values are tracked at different training iterations for the clusters identified as challenging and prioritized during the Focus stage of LFR. The dotted line represents the PPL values for the same clusters when trained with random sampling (RS). Notably, LFR facilitates accelerated learning of these challenging data points between 20k and 60k iterations (the Focus stage), whereas random sampling consistently results in higher PPL values throughout.
• Figure 5: PPL values are tracked at different training iterations for the clusters identified as easy, discarded during the Focus stage, and reintroduced during the Review phase. The dotted line represents the PPL values for the same clusters when trained with random sampling (RS). Notably, we demonstrate that models forget the data points discarded during training, unless reintroduced to the training corpus as in the case of LFR.