Potential Energy based Mixture Model for Noisy Label Learning

TL;DR

The paper tackles learning with noisy labels in deep networks by leveraging inherent data structure through a latent mixture-model view. It introduces PEMM, a distance-based classifier whose class centers are regularized by a potential-energy term, promoting a co-stable geometric arrangement that reduces reliance on noisy labels. The approach is implemented on standard backbones and shows strong robustness on CIFAR-10/100 with synthetic noise and Clothing1M, alongside theoretical framing that links data structure to learning objectives. Its simplicity and compatibility with existing models suggest wide practical applicability for robust learning under label noise.

Abstract

Training deep neural networks (DNNs) from noisy labels is an important and challenging task. However, most existing approaches focus on the corrupted labels and ignore the importance of inherent data structure. To bridge the gap between noisy labels and data, inspired by the concept of potential energy in physics, we propose a novel Potential Energy based Mixture Model (PEMM) for noise-labels learning. We innovate a distance-based classifier with the potential energy regularization on its class centers. Embedding our proposed classifier with existing deep learning backbones, we can have robust networks with better feature representations. They can preserve intrinsic structures from the data, resulting in a superior noisy tolerance. We conducted extensive experiments to analyze the efficiency of our proposed model on several real-world datasets. Quantitative results show that it can achieve state-of-the-art performance.

Figures (8)

• Figure 1: An illustration of potential energy. The energy of the molecule is a function of the distance between the two nuclei. The most stable state(potential energy is -436KJ/mol) is achieved at the distance 74 pm.
• Figure 2: The framework for noisy robust learning.
• Figure 3: The establishment process of co-stable state: starting from random positions, then each node will look for a stable position automatically.
• Figure 4: Illustration of the difference between the loss in chen2019joint(red) and our PE loss(black). The PE loss value is asymmetric.
• Figure 5: Change process of a co-stable process.(left to right)