Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncertainty and Energy based Loss Guided Semi-Supervised Semantic Segmentation

Rini Smita Thakur, Vinod K. Kurmi

TL;DR

This work addresses the challenge of semi-supervised semantic segmentation under limited labeled data by integrating aleatoric data uncertainty and energy-based modeling into a two-branch union-intersection pseudo-label framework (DUEB) built on CPCL and DeepLabv3+. The approach couples per-pixel variance estimation with an energy-based loss, applied to both conservative and progressive branches, to refine pseudo-labels and improve calibration. Empirical results on Cityscapes and PASCAL VOC show consistent gains, especially at low-label regimes, and ablations confirm that combining uncertainty and energy losses yields the strongest improvements. The method offers a practical pathway to improve robustness and generalization in SS segmentation and can be extended to transformer-based architectures.

Abstract

Semi-supervised (SS) semantic segmentation exploits both labeled and unlabeled images to overcome tedious and costly pixel-level annotation problems. Pseudolabel supervision is one of the core approaches of training networks with both pseudo labels and ground-truth labels. This work uses aleatoric or data uncertainty and energy based modeling in intersection-union pseudo supervised network.The aleatoric uncertainty is modeling the inherent noise variations of the data in a network with two predictive branches. The per-pixel variance parameter obtained from the network gives a quantitative idea about the data uncertainty. Moreover, energy-based loss realizes the potential of generative modeling on the downstream SS segmentation task. The aleatoric and energy loss are applied in conjunction with pseudo-intersection labels, pseudo-union labels, and ground-truth on the respective network branch. The comparative analysis with state-of-the-art methods has shown improvement in performance metrics.

Uncertainty and Energy based Loss Guided Semi-Supervised Semantic Segmentation

TL;DR

This work addresses the challenge of semi-supervised semantic segmentation under limited labeled data by integrating aleatoric data uncertainty and energy-based modeling into a two-branch union-intersection pseudo-label framework (DUEB) built on CPCL and DeepLabv3+. The approach couples per-pixel variance estimation with an energy-based loss, applied to both conservative and progressive branches, to refine pseudo-labels and improve calibration. Empirical results on Cityscapes and PASCAL VOC show consistent gains, especially at low-label regimes, and ablations confirm that combining uncertainty and energy losses yields the strongest improvements. The method offers a practical pathway to improve robustness and generalization in SS segmentation and can be extended to transformer-based architectures.

Abstract

Semi-supervised (SS) semantic segmentation exploits both labeled and unlabeled images to overcome tedious and costly pixel-level annotation problems. Pseudolabel supervision is one of the core approaches of training networks with both pseudo labels and ground-truth labels. This work uses aleatoric or data uncertainty and energy based modeling in intersection-union pseudo supervised network.The aleatoric uncertainty is modeling the inherent noise variations of the data in a network with two predictive branches. The per-pixel variance parameter obtained from the network gives a quantitative idea about the data uncertainty. Moreover, energy-based loss realizes the potential of generative modeling on the downstream SS segmentation task. The aleatoric and energy loss are applied in conjunction with pseudo-intersection labels, pseudo-union labels, and ground-truth on the respective network branch. The comparative analysis with state-of-the-art methods has shown improvement in performance metrics.
Paper Structure (18 sections, 17 equations, 3 figures, 5 tables)

This paper contains 18 sections, 17 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Semi-supervised segmentation
  4. Uncertainty Estimation
  5. Method
  6. Problem Statement
  7. Network Architecture
  8. Data Uncertainty Estimation
  9. Energy Modeling
  10. Total loss
  11. Experiments
  12. Dataset and Implementation Details
  13. Datasets
  14. Implementation Details
  15. Performance Analysis
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Refinement of pseudolabel with aleatoric uncertainty and energy based modeling
  • Figure 2: Block Diagram of proposed uncertainty and energy loss based framework with pseudolabels module for semi-supervised segmentation. Two branches: conservative and progressive are trained in parallel with intersection/union pseudolabels, data uncertainty loss and energy based loss.
  • Figure 3: Segmentation Results of Cityscapes dataset (partition protocol:1/8) Left:Input Middle:Predictions of DUEB Right:Ground-truth.