Sm: enhanced localization in Multiple Instance Learning for medical imaging classification

TL;DR

This work proposes a novel, principled, and flexible mechanism to model local dependencies in multiple Instance Learning that leads to state-of-the-art performance in localization while being competitive or superior in classification.

Abstract

Multiple Instance Learning (MIL) is widely used in medical imaging classification to reduce the labeling effort. While only bag labels are available for training, one typically seeks predictions at both bag and instance levels (classification and localization tasks, respectively). Early MIL methods treated the instances in a bag independently. Recent methods account for global and local dependencies among instances. Although they have yielded excellent results in classification, their performance in terms of localization is comparatively limited. We argue that these models have been designed to target the classification task, while implications at the instance level have not been deeply investigated. Motivated by a simple observation -- that neighboring instances are likely to have the same label -- we propose a novel, principled, and flexible mechanism to model local dependencies. It can be used alone or combined with any mechanism to model global dependencies (e.g., transformers). A thorough empirical validation shows that our module leads to state-of-the-art performance in localization while being competitive or superior in classification. Our code is at https://github.com/Franblueee/SmMIL.

Figures (14)

• Figure 1: (a) Unified view of deep MIL models. Depending on how instances interact with each other in (a), we devise three different families of methods: (b), (c), (d).
• Figure 2: WSIs are divided into patches. CT scans are provided as slices. They often show spatial dependencies: in a WSI, a patch is usually surrounded by patches with the same label, while in a CT scan, a slice is usually surrounded by slices with the same label. The red color indicates malignant/hemorrhage patches/slices.
• Figure 3: Smooth Attention Multiple Instance Learning. (a) The well-known model in ilse2018attention, which we build upon. (b): only local interactions are considered by applying the proposed smooth operator Sm in the aggregation part. (c): both global and local interactions are considered by applying Sm both in the transformer and in the aggregation parts.
• Figure 4: Attention histograms on CAMELYON16. First/second rows show models without/with global interactions. Sm AP and Sm TAP stand out at separating positive and negative instances.
• Figure 5: Attention maps on CAMELYON16. The novel Sm TAP produces the most accurate one.