Learning from Noisy Labels via Self-Taught On-the-Fly Meta Loss Rescaling

TL;DR

STORM tackles the pervasive problem of noisy labels by proposing a self-taught on-the-fly meta loss rescaling framework that does not require clean validation data. It learns a lightweight rescaling function Omega through meta-learning, using features derived from the model's own predictions and losses, and updates both the model Theta and the rescaler jointly in a single training loop. Empirically, STORM improves performance across diverse NLP tasks, including dialogue state tracking with TripPy, while maintaining or enhancing calibration and showing resilience to class imbalance and various noise types. The approach offers practical benefits by avoiding data loss from hard filtering, reducing overfitting to noise, and providing a scalable, data-driven way to leverage noisy data in real-world settings.

Abstract

Correct labels are indispensable for training effective machine learning models. However, creating high-quality labels is expensive, and even professionally labeled data contains errors and ambiguities. Filtering and denoising can be applied to curate labeled data prior to training, at the cost of additional processing and loss of information. An alternative is on-the-fly sample reweighting during the training process to decrease the negative impact of incorrect or ambiguous labels, but this typically requires clean seed data. In this work we propose unsupervised on-the-fly meta loss rescaling to reweight training samples. Crucially, we rely only on features provided by the model being trained, to learn a rescaling function in real time without knowledge of the true clean data distribution. We achieve this via a novel meta learning setup that samples validation data for the meta update directly from the noisy training corpus by employing the rescaling function being trained. Our proposed method consistently improves performance across various NLP tasks with minimal computational overhead. Further, we are among the first to attempt on-the-fly training data reweighting on the challenging task of dialogue modeling, where noisy and ambiguous labels are common. Our strategy is robust in the face of noisy and clean data, handles class imbalance, and prevents overfitting to noisy labels. Our self-taught loss rescaling improves as the model trains, showing the ability to keep learning from the model's own signals. As training progresses, the impact of correctly labeled data is scaled up, while the impact of wrongly labeled data is suppressed.

Figures (8)

• Figure 1: Schematic of STORM. Numbered black arrows correspond to the inner loop updates, i.e., learning $\Theta$. Numbered red and blue arrows correspond to the outer loop updates, i.e., meta learning $\Omega$. Dashed arrows indicate gradient flows.
• Figure 3: #1: A wrongly labeled sample is discarded consistently after initial consideration in the first epoch. #2: An ambiguous sample is initially ignored, then gradually upscaled. #3: A clean but hard sample sees its loss weight increased steadily from mid-training. #4: A clean and easy sample has its high initial loss weight reduced due to decreasing informativeness.
• Figure : (a) Rescaling ROC
• Figure : (a) Rescaling ROC
• Figure : (b) Filter timing