Reverse Modeling in Large Language Models
Sicheng Yu, Yuanchen Xu, Cunxiao Du, Yanying Zhou, Minghui Qiu, Qianru Sun, Hao Zhang, Jiawei Wu
TL;DR
This work probes whether LLMs can perform reverse modeling with reversed inputs and finds that publicly available pre-trained models exhibit a forward-direction bias, whereas models trained from scratch with both forward and reverse data learn to process reverse inputs across languages. The authors introduce a loss-difference signal $S$ = Avg Forward Loss - Avg Reverse Loss to quantify data quality and guide reverse-informed data selection during pretraining, showing that selecting high-$S$ data yields substantial gains on language-understanding benchmarks such as MMLU across multiple backbones. Across multilingual setups and backbones, the study demonstrates that reverse-favoring data tend to be higher quality and more coherent, enabling improved knowledge acquisition. While promising, the approach relies on a simplistic reversal of token sequences to simulate reverse thinking and hinges on evaluation metrics that may not capture all aspects of reverse reasoning or computation efficiency; future work should explore more nuanced reverse-thinking simulations and broader benchmarks to validate and extend these findings.
Abstract
Humans are accustomed to reading and writing in a forward manner, and this natural bias extends to text understanding in auto-regressive large language models (LLMs). This paper investigates whether LLMs, like humans, struggle with reverse modeling, specifically with reversed text inputs. We found that publicly available pre-trained LLMs cannot understand such inputs. However, LLMs trained from scratch with both forward and reverse texts can understand them equally well during inference across multiple languages. Our case study shows that different-content texts result in different losses if input (to LLMs) in different directions -- some get lower losses for forward while some for reverse. This leads us to a simple and nice solution for data selection based on the loss differences between forward and reverse directions. Using our selected data in continued pretraining can boost LLMs' performance by a large margin across different language understanding benchmarks.