Hierarchical Superpixel Segmentation via Structural Information Theory

TL;DR

The paper tackles the limitation of traditional graph-based superpixel methods that rely on local, adjacent pixel relationships by introducing SIT-HSS, an interpretable hierarchical segmentation framework grounded in structural information theory. It constructs a pixel graph by maximizing the 1D structural entropy $H^{(1)}_r(G)$ over increasing radii to retain imaging information without overcomplicating the graph, followed by a hierarchical merging strategy that minimizes the 2D entropy $H^{(2)}_{igl\P\bigr}(G)$ to reach a predefined scale $K$. Key contributions include (1) a formal SIT framework based on encoding trees, (2) a novel $1$D SE–maximizing graph construction, and (3) a $2$D SE–driven merging procedure, validated by extensive experiments on BSDS500, SBD, and PASCAL-S that achieve state-of-the-art unsupervised performance with competitive efficiency. The work provides both quantitative gains and qualitative interpretability through visualizations of the graph, encoding trees, and segmentation results, and the code is released for reproducibility and reuse.

Abstract

Superpixel segmentation is a foundation for many higher-level computer vision tasks, such as image segmentation, object recognition, and scene understanding. Existing graph-based superpixel segmentation methods typically concentrate on the relationships between a given pixel and its directly adjacent pixels while overlooking the influence of non-adjacent pixels. These approaches do not fully leverage the global information in the graph, leading to suboptimal segmentation quality. To address this limitation, we present SIT-HSS, a hierarchical superpixel segmentation method based on structural information theory. Specifically, we first design a novel graph construction strategy that incrementally explores the pixel neighborhood to add edges based on 1-dimensional structural entropy (1D SE). This strategy maximizes the retention of graph information while avoiding an overly complex graph structure. Then, we design a new 2D SE-guided hierarchical graph partitioning method, which iteratively merges pixel clusters layer by layer to reduce the graph's 2D SE until a predefined segmentation scale is achieved. Experimental results on three benchmark datasets demonstrate that the SIT-HSS performs better than state-of-the-art unsupervised superpixel segmentation algorithms. The source code is available at \url{https://github.com/SELGroup/SIT-HSS}.

Figures (11)

• Figure 1: An illustration of superpixel segmentation using SIT-HSS. It performs superpixel segmentation by first constructing a weighted graph, then obtaining an optimal encoding tree, and finally achieving multi-scale superpixel segmentation.
• Figure 2: The framework of the SIT-HSS. I and II represent the weighted graph construction and partitioning processes, respectively. We add edges to the edgeless pixel graph by progressively expanding the neighbourhood. Then, for each superpixel, we iteratively merge neighbouring superpixel pairs that maximally reduce the 2D SE until the target size is reached.
• Figure 4: ASA, BR, UE, and EV Curve of our SIT-HSS compared with other state-of-the-art superpixel segmentation algorithms on the BSDS500 dataset.
• Figure 5: Visualization of different superpixel segmentation algorithms on different datasets when $K = 100$.
• Figure 6: Visualization of the effectiveness validation of graph construction and partitioning.

Theorems & Definitions (2)

• Definition 1
• Definition 2