The Re-Label Method For Data-Centric Machine Learning

TL;DR

The paper tackles the issue of noisy labels in manually labeled industrial DL data and proposes a data-centric re-label method that uses model predictions as labeling references for humans. It applies this iterative relabeling across tasks including text classification, sequence tagging, object detection, sequence generation, and CTR prediction, detailing task-specific adaptations. Experimental results on dev datasets show notable improvements in accuracy and human-evaluated quality for the relabeled data and updated models. The work highlights the value of a human-in-the-loop approach to data cleansing, with potential to reduce labeling costs while boosting robustness in real-world deployments.

Abstract

In industry deep learning application, our manually labeled data has a certain number of noisy data. To solve this problem and achieve more than 90 score in dev dataset, we present a simple method to find the noisy data and re-label the noisy data by human, given the model predictions as references in human labeling. In this paper, we illustrate our idea for a broad set of deep learning tasks, includes classification, sequence tagging, object detection, sequence generation, click-through rate prediction. The dev dataset evaluation results and human evaluation results verify our idea.

Figures (1)

• Figure 1: Re-label method applying to any machine learning application.