Incremental Sequence Classification with Temporal Consistency
Lucas Maystre, Gabriel Barello, Tudor Berariu, Aleix Cambray, Rares Dolga, Alvaro Ortega Gonzalez, Andrei Nica, David Barber
TL;DR
The paper introduces a temporally-consistent, TD-inspired loss (TC-$\lambda$) for incremental sequence classification, enabling predictions at every prefix with a single model. It establishes theoretical convergence and consistency in a tabular setting and demonstrates data-efficiency advantages over direct cross-entropy, both in theory and in practice. Empirically, TC-$\lambda$ improves prefix and full-sequence accuracy on text classification and yields stronger early correctness signals for GSM8K verification, enabling compute-aware inference strategies. The approach is architecture-agnostic, shows promise for scalable verification and multi-token prediction, and points to meaningful practical benefits in real-time decision-making and LLM evaluation, while noting the need for broader evaluations on larger models and multimodal tasks.
Abstract
We address the problem of incremental sequence classification, where predictions are updated as new elements in the sequence are revealed. Drawing on temporal-difference learning from reinforcement learning, we identify a temporal-consistency condition that successive predictions should satisfy. We leverage this condition to develop a novel loss function for training incremental sequence classifiers. Through a concrete example, we demonstrate that optimizing this loss can offer substantial gains in data efficiency. We apply our method to text classification tasks and show that it improves predictive accuracy over competing approaches on several benchmark datasets. We further evaluate our approach on the task of verifying large language model generations for correctness in grade-school math problems. Our results show that models trained with our method are better able to distinguish promising generations from unpromising ones after observing only a few tokens.