Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PDL: Regularizing Multiple Instance Learning with Progressive Dropout Layers

Wenhui Zhu, Peijie Qiu, Xiwen Chen, Oana M. Dumitrascu, Yalin Wang

TL;DR

MIL models suffer from overfitting due to weak supervision. The paper presents Progressive Dropout Layer (PDL), a MIL-specific dropout with attention-based instance weighting and a progressive scheduler that regularizes the middle layer without altering MIL architectures. Across standard MIL benchmarks and WSIs, PDL yields consistent improvements in accuracy and localization, and analyses reveal effective drop-rate control and the scheduler's necessity. The work offers a practical, plug-in regularization tool that enhances weakly supervised learning and latent feature discovery in MIL.

Abstract

Multiple instance learning (MIL) was a weakly supervised learning approach that sought to assign binary class labels to collections of instances known as bags. However, due to their weak supervision nature, the MIL methods were susceptible to overfitting and required assistance in developing comprehensive representations of target instances. While regularization typically effectively combated overfitting, its integration with the MIL model has been frequently overlooked in prior studies. Meanwhile, current regularization methods for MIL have shown limitations in their capacity to uncover a diverse array of representations. In this study, we delve into the realm of regularization within the MIL model, presenting a novel approach in the form of a Progressive Dropout Layer (PDL). We aim to not only address overfitting but also empower the MIL model in uncovering intricate and impactful feature representations. The proposed method was orthogonal to existing MIL methods and could be easily integrated into them to boost performance. Our extensive evaluation across a range of MIL benchmark datasets demonstrated that the incorporation of the PDL into multiple MIL methods not only elevated their classification performance but also augmented their potential for weakly-supervised feature localizations.

PDL: Regularizing Multiple Instance Learning with Progressive Dropout Layers

TL;DR

MIL models suffer from overfitting due to weak supervision. The paper presents Progressive Dropout Layer (PDL), a MIL-specific dropout with attention-based instance weighting and a progressive scheduler that regularizes the middle layer without altering MIL architectures. Across standard MIL benchmarks and WSIs, PDL yields consistent improvements in accuracy and localization, and analyses reveal effective drop-rate control and the scheduler's necessity. The work offers a practical, plug-in regularization tool that enhances weakly supervised learning and latent feature discovery in MIL.

Abstract

Multiple instance learning (MIL) was a weakly supervised learning approach that sought to assign binary class labels to collections of instances known as bags. However, due to their weak supervision nature, the MIL methods were susceptible to overfitting and required assistance in developing comprehensive representations of target instances. While regularization typically effectively combated overfitting, its integration with the MIL model has been frequently overlooked in prior studies. Meanwhile, current regularization methods for MIL have shown limitations in their capacity to uncover a diverse array of representations. In this study, we delve into the realm of regularization within the MIL model, presenting a novel approach in the form of a Progressive Dropout Layer (PDL). We aim to not only address overfitting but also empower the MIL model in uncovering intricate and impactful feature representations. The proposed method was orthogonal to existing MIL methods and could be easily integrated into them to boost performance. Our extensive evaluation across a range of MIL benchmark datasets demonstrated that the incorporation of the PDL into multiple MIL methods not only elevated their classification performance but also augmented their potential for weakly-supervised feature localizations.
Paper Structure (45 sections, 8 equations, 7 figures, 5 tables)

This paper contains 45 sections, 8 equations, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. MIL methods:
  4. Dropout methods:
  5. MIL Preliminaries and Problem Statement
  6. MIL Preliminaries
  7. Instance-level projector.
  8. MIL pooling.
  9. Problem Statement
  10. Progressive Dropout Layer
  11. MIL-Specific Attention-Based Dropout
  12. Avarage-pooling based attention.
  13. Dynamic drop rate assignment to each instance.
  14. Instance-based dropout.
  15. Progressive Learning Scheduler
  16. ...and 30 more sections

Figures (7)

  • Figure 1: (A). The MI-Net utilized the MIL pooling to enhance supervision across various embedding instance dimensions. (B). Illustration of progressive dropout layer (PDL) mechanism under the MIL workflow, taking one layer as an example. PDL dynamically assigned a drop rate to each instance based on Avarage-Pooling Based Attention (APBA), and applied Instance-Based Dropout to instances with drop rate $p'_{k}$. The Progressive Learning Scheduler controlled the global maximum drop rate $P$ to generate a set of $\left \{ p_{1}, p_{2}, \dots, p_{k} \right \}$. This mechanism enforced the model to discover latent features and mitigate overfitting.
  • Figure 2: The Camelyon16 experiment loss visualization before and after the integration of PDL during training.
  • Figure 3: Tumor localization in WSIs comparing ABMIL with PDL and Without PDL based on Camlyon16.
  • Figure 4: The APBA visualization of PDL in training, the number represents the attention weight, and the red block denotes the assigned top three drop rate.
  • Figure 5: The attention localization of ABMIL.
  • ...and 2 more figures

Theorems & Definitions (1)

  • Definition 1