Prioritizing Informative Features and Examples for Deep Learning from Noisy Data

TL;DR

This work tackles robust deep learning in the presence of noisy data by proposing a data-centric framework that prioritizes informative information across three development stages. It introduces TAUFE, a task-agnostic feature-level calibration that deactivates undesirable out-of-distribution features at the extractor level, and MQNet, a meta-model that balances purity and informativeness in open-set active learning to reduce labeling costs. It further develops Prune4ReL for robust data pruning under label noise and FP-Instruction for selecting clean, diverse, and high-quality instructions to improve LLM factuality and user-preferred outputs. Across comprehensive experiments on image classification, detection/localization tasks, active learning benchmarks, and instruction tuning, the approach demonstrates robustness to noise and scalability, offering practical improvements for real-world noisy-data pipelines with theoretical underpinnings. Overall, the unified framework advances robust, data-centric deep learning workflows by mitigating both noisy features and noisy samples throughout model development and deployment.

Abstract

In this dissertation, we propose a systemic framework that prioritizes informative features and examples to enhance each stage of the development process. Specifically, we prioritize informative features and examples and improve the performance of feature learning, data labeling, and data selection. We first propose an approach to extract only informative features that are inherent to solving a target task by using auxiliary out-of-distribution data. We deactivate the noise features in the target distribution by using that in the out-of-distribution data. Next, we introduce an approach that prioritizes informative examples from unlabeled noisy data in order to reduce the labeling cost of active learning. In order to solve the purity-information dilemma, where an attempt to select informative examples induces the selection of many noisy examples, we propose a meta-model that finds the best balance between purity and informativeness. Lastly, we suggest an approach that prioritizes informative examples from labeled noisy data to preserve the performance of data selection. For labeled image noise data, we propose a data selection method that considers the confidence of neighboring samples to maintain the performance of the state-of-the-art Re-labeling models. For labeled text noise data, we present an instruction selection method that takes diversity into account for ranking the quality of instructions with prompting, thereby enhancing the performance of aligned large language models. Overall, our unified framework induces the deep learning development process robust to noisy data, thereby effectively mitigating noisy features and examples in real-world applications.

Figures (20)

• Figure 1: Informative/noisy features and examples.
• Figure 2: Negative effect of noisy features and examples throughout model development process, and our solution.
• Figure 3: Comparison of softmax-level and feature-level calibrations.
• Figure 4: Effect of the softmax-level and feature-level calibrations on the penultimate layer activations.
• Figure 5: TSNE visualization of the penultimate layer activations. In-distribution examples are in pink for the automobile class and in blue for the bird class, while all OOD examples are in grey.

Theorems & Definitions (24)

• Definition 3.2.1
• Definition 3.2.2
• Theorem 4.3.1
• proof
• Lemma 4.3.2
• proof
• Lemma 4.3.3
• proof
• Lemma 4.3.4
• proof