Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Precision matters: Precision-aware ensemble for weakly supervised semantic segmentation

Junsung Park, Hyunjung Shim

TL;DR

The paper addresses the mismatch between pseudo-mask quality measured by mIoU and actual segmentation performance in weakly supervised semantic segmentation. It introduces ORANDNet, a precision-focused CAM ensemble that fuses two diverse CAMs via pixel-wise OR and AND operations and refines the result with an FCN, followed by IRN processing; scale scheduling further reduces noise early in training. Experiments on Pascal VOC 2012 show that ORANDNet improves both mIoU and precision over single backbones and naïve ensembles, and can extend to advanced WSSS models like AMN and MCTFormer, highlighting its potential as a versatile add-on module. Overall, the approach provides a simple, effective, and broadly applicable enhancement for WSSS pipelines, emphasizing precision to achieve better final segmentation.

Abstract

Weakly Supervised Semantic Segmentation (WSSS) employs weak supervision, such as image-level labels, to train the segmentation model. Despite the impressive achievement in recent WSSS methods, we identify that introducing weak labels with high mean Intersection of Union (mIoU) does not guarantee high segmentation performance. Existing studies have emphasized the importance of prioritizing precision and reducing noise to improve overall performance. In the same vein, we propose ORANDNet, an advanced ensemble approach tailored for WSSS. ORANDNet combines Class Activation Maps (CAMs) from two different classifiers to increase the precision of pseudo-masks (PMs). To further mitigate small noise in the PMs, we incorporate curriculum learning. This involves training the segmentation model initially with pairs of smaller-sized images and corresponding PMs, gradually transitioning to the original-sized pairs. By combining the original CAMs of ResNet-50 and ViT, we significantly improve the segmentation performance over the single-best model and the naive ensemble model, respectively. We further extend our ensemble method to CAMs from AMN (ResNet-like) and MCTformer (ViT-like) models, achieving performance benefits in advanced WSSS models. It highlights the potential of our ORANDNet as a final add-on module for WSSS models.

Precision matters: Precision-aware ensemble for weakly supervised semantic segmentation

TL;DR

The paper addresses the mismatch between pseudo-mask quality measured by mIoU and actual segmentation performance in weakly supervised semantic segmentation. It introduces ORANDNet, a precision-focused CAM ensemble that fuses two diverse CAMs via pixel-wise OR and AND operations and refines the result with an FCN, followed by IRN processing; scale scheduling further reduces noise early in training. Experiments on Pascal VOC 2012 show that ORANDNet improves both mIoU and precision over single backbones and naïve ensembles, and can extend to advanced WSSS models like AMN and MCTFormer, highlighting its potential as a versatile add-on module. Overall, the approach provides a simple, effective, and broadly applicable enhancement for WSSS pipelines, emphasizing precision to achieve better final segmentation.

Abstract

Weakly Supervised Semantic Segmentation (WSSS) employs weak supervision, such as image-level labels, to train the segmentation model. Despite the impressive achievement in recent WSSS methods, we identify that introducing weak labels with high mean Intersection of Union (mIoU) does not guarantee high segmentation performance. Existing studies have emphasized the importance of prioritizing precision and reducing noise to improve overall performance. In the same vein, we propose ORANDNet, an advanced ensemble approach tailored for WSSS. ORANDNet combines Class Activation Maps (CAMs) from two different classifiers to increase the precision of pseudo-masks (PMs). To further mitigate small noise in the PMs, we incorporate curriculum learning. This involves training the segmentation model initially with pairs of smaller-sized images and corresponding PMs, gradually transitioning to the original-sized pairs. By combining the original CAMs of ResNet-50 and ViT, we significantly improve the segmentation performance over the single-best model and the naive ensemble model, respectively. We further extend our ensemble method to CAMs from AMN (ResNet-like) and MCTformer (ViT-like) models, achieving performance benefits in advanced WSSS models. It highlights the potential of our ORANDNet as a final add-on module for WSSS models.
Paper Structure (12 sections, 1 equation, 3 figures, 2 tables)

This paper contains 12 sections, 1 equation, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methods
  4. Motivation
  5. Proposed method
  6. ORANDNet
  7. Scale scheduling
  8. Experiments
  9. Ablation study
  10. Comparing segmentation results with recent WSSS
  11. Conclusion
  12. Achknowledgement

Figures (3)

  • Figure 1: (a) Difference between ResNet-based CAM and ViT-based CAM. "Ours" and "Ours*" indicate the ORANDNet ensemble of ResNet-50-ViT and AMN-MCTformer, respectively. (b) Visualization of OR CAM (1st row) and AND CAM (2nd row).
  • Figure 2: The overall pipeline of our method includes: (a) ORANDNet during training, and (b) ORANDNet in the test time with scale scheduling. $\mathcal{L}_{\textit{ce}}$ and $\mathcal{L}_{\textit{ls, bce}}$ mean cross entropy and class-balanced cross entropy with label smoothing, respectively.
  • Figure 3: Segmentation results of AMN and MCTformer and their ORANDNet ensemble.