Uncertainty-Participation Context Consistency Learning for Semi-supervised Semantic Segmentation

TL;DR

The paper addresses the reliance on high-certainty pixels in semi-supervised semantic segmentation by introducing Uncertainty-participation Context Consistency Learning (UCCL). It jointly leverages uncertain pixel regions via Semantic Backpropagation Update (SBU) and enforces class-level consistency across augmentations with Class-aware Knowledge Regulation (CKR), integrated into a loss framework $ abla \mathcal{L} = \mathcal{L}_s + \mathcal{L}_x + \alpha \mathcal{L}_{su} + \beta \mathcal{L}_{cr}$. Empirical results on Pascal VOC2012 and Cityscapes demonstrate state-of-the-art performance across varying labeled/unlabeled splits, validating the approach's effectiveness in extracting richer supervisory signals. The work highlights the practical impact of exploiting uncertain regions and class-level context to enhance encoder robustness and segmentation accuracy, and provides a reusable codebase for the community. The key innovations are the SBU mechanism that weights supervision by cross-view pixel similarities in uncertain regions and the CKR module that aligns class-level semantic features across augmentations.

Abstract

Semi-supervised semantic segmentation has attracted considerable attention for its ability to mitigate the reliance on extensive labeled data. However, existing consistency regularization methods only utilize high certain pixels with prediction confidence surpassing a fixed threshold for training, failing to fully leverage the potential supervisory information within the network. Therefore, this paper proposes the Uncertainty-participation Context Consistency Learning (UCCL) method to explore richer supervisory signals. Specifically, we first design the semantic backpropagation update (SBU) strategy to fully exploit the knowledge from uncertain pixel regions, enabling the model to learn consistent pixel-level semantic information from those areas. Furthermore, we propose the class-aware knowledge regulation (CKR) module to facilitate the regulation of class-level semantic features across different augmented views, promoting consistent learning of class-level semantic information within the encoder. Experimental results on two public benchmarks demonstrate that our proposed method achieves state-of-the-art performance. Our code is available at https://github.com/YUKEKEJAN/UCCL.

Figures (3)

• Figure 1: Comparison with previous methods. (a) The existing methods exclusively leverage labels of high certain pixels (non-white regions in $p^w$) for supervised training to ensure consistency in prediction results ($p^s$ and $p^w$). (b) Our approach focuses more on utilizing information from uncertain pixels (white regions in $p^w$) in strongly and weakly augmented views ($x^s$ and $x^w$) by semantic backpropagation update (SBU), while also exploring potential consistent semantic information via class-aware knowledge regulation (CKR).
• Figure 2: Illustration of our UCCL method. The paired encoder and decoder parameters are shared.
• Figure 3: Comparison of visualization results with state-of-the-art methods under the Full (1464) setting on the Pascal VOC2012 dataset.