Noise-Tolerant Hybrid Prototypical Learning with Noisy Web Data

TL;DR

The paper tackles learning unbiased few-shot classifiers using a small clean set and a large noisy web data pool. It introduces SimNoiPro, a similarity-maximization loss that jointly learns noise-tolerant hybrid prototypes by partitioning noisy samples into multiple groups and align­ing them with the clean prototype in an end-to-end framework. This approach overcomes the limitations of single noise prototypes and binary relevance losses, yielding more compact and discriminative class prototypes and improved performance on Low-shot Places365 and Low-shot ImageNet. The method demonstrates robust gains in low-shot regimes and offers an effective strategy for leveraging noisy web data in practical few-shot learning scenarios.

Abstract

We focus on the challenging problem of learning an unbiased classifier from a large number of potentially relevant but noisily labeled web images given only a few clean labeled images. This problem is particularly practical because it reduces the expensive annotation costs by utilizing freely accessible web images with noisy labels. Typically, prototypes are representative images or features used to classify or identify other images. However, in the few clean and many noisy scenarios, the class prototype can be severely biased due to the presence of irrelevant noisy images. The resulting prototypes are less compact and discriminative, as previous methods do not take into account the diverse range of images in the noisy web image collections. On the other hand, the relation modeling between noisy and clean images is not learned for the class prototype generation in an end-to-end manner, which results in a suboptimal class prototype. In this article, we introduce a similarity maximization loss named SimNoiPro. Our SimNoiPro first generates noise-tolerant hybrid prototypes composed of clean and noise-tolerant prototypes and then pulls them closer to each other. Our approach considers the diversity of noisy images by explicit division and overcomes the optimization discrepancy issue. This enables better relation modeling between clean and noisy images and helps extract judicious information from the noisy image set. The evaluation results on two extended few-shot classification benchmarks confirm that our SimNoiPro outperforms prior methods in measuring image relations and cleaning noisy data.

Figures (8)

• Figure 1: Relevant images exist in web images. Massive freely accessible web images can be obtained by search engines. These images are easily acquired but can be inevitably annotated with noisy labels.
• Figure 2: Overall framework. First, SimNoiPro divides noisy examples into several groups by class relevance scores $r$ to produce noise-tolerant prototypes. Then, the similarity maximization loss pulls noise-tolerant prototypes closer to the clean prototype in order to generate a more discriminative prototype for classification.
• Figure 3: Effect of the number of noise-tolerant prototypes. More noise-tolerant prototypes can lead to better performance before saturation.
• Figure 4: Effect of the hyperparameter setting in our SimNoiPro loss. Both global and local terms are important. Increasing $\alpha_t$ is preferred.
• Figure 5: Class relevance $r$ distribution comparison on Low-shot Places365. (a) is for class 2. (b) is for class 10. (c) is for class 360. Our SimNoiPro reveals the relative importance of noisy features.