AINet+: Advancing Superpixel Segmentation via Cascaded Association Implantation

TL;DR

The paper tackles the limitation of conventional CNN-based superpixel methods that rely on restricted receptive fields, hindering explicit modeling of pixel-grid interactions. It introduces Association Implantation (AI), which embeds grid features around each pixel and applies a 3×3 convolution to distill pixel–grid context, enabling progressive refinement through hierarchical association learning and a boundary-perceiving loss to sharpen boundary delineation. The proposed AINet+ architecture combines AI at multiple layers and a boundary-focused objective, achieving state-of-the-art performance on BSDS500, NYUv2, ISIC-2017, and ACDC, while also improving downstream tasks such as object proposal generation and stereo matching. This approach provides a principled, explicit pixel-grid modeling framework for superpixel segmentation with strong cross-domain generalization and practical downstream impact.

Abstract

Superpixel segmentation has seen significant progress benefiting from the deep convolutional networks. The typical approach entails initial division of the image into grids, followed by a learning process that assigns each pixel to adjacent grid segments. However, reliance on convolutions with confined receptive fields results in an implicit, rather than explicit, understanding of pixel-grid interactions. This limitation often leads to a deficit of contextual information during the mapping of associations. To counteract this, we introduce the Association Implantation (AI) module, designed to allow networks to explicitly engage with pixel-grid relationships. This module embeds grid features directly into the vicinity of the central pixel and employs convolutional operations on an enlarged window, facilitating an adaptive transfer of knowledge. This approach enables the network to explicitly extract context at the pixel-grid level, which is more aligned with the objectives of superpixel segmentation than mere pixel-wise interactions. By integrating the AI module across various layers, we enable a progressive refinement of pixel-superpixel relationships from coarse to fine. To further enhance the assignment of boundary pixels, we've engineered a boundary-aware loss function. This function aids in the discrimination of boundary-adjacent pixels at the feature level, thereby empowering subsequent modules to precisely identify boundary pixels and enhance overall boundary accuracy. Our method has been rigorously tested on four benchmarks, including BSDS500, NYUv2, ACDC, and ISIC2017, and our model can achieve competitive performance with comparison methods.

Figures (14)

• Figure 1: Different from the SCN SCN that implicitly learns the association using the stacked naı ve convolutions, we proposes to implant the corresponding grid features to the surrounding of the pixel to explicitly perceive the relation between each pixel and its neighbor grids.
• Figure 2: The framework of our AINet+, where the conv/deconv means convolution/deconvolution, and conv-S# means convolution with stride #. The network takes an image as input and outputs the association map. Meanwhile, the superpixel embedding and pixel embedding are first obtained by the convolutions and then fed into the AI module to obtain the pixel-superpixel context. And the local patch loss is performed on the pixel-wise embeddings to boost the boundary precision. In AI module, the sampling interval is set to 16, and each block indicates a pixel or superpixel embedding.
• Figure 3: Illustration of our Association Implantation Module (AI). We directly implant the surrounding grid features to the central pixels, such a design could allow the network to explicitly harvest the pixel-grid relation, which is the context exactly required by the superpixel segmentation.
• Figure 4: Illustration of our Boundary-perceiving Loss (BPL). By performing a classification procedure on the boundary patches, we targets at discriminating the boundary pixels in hidden feature level, which is achieved by enforcing the pixels with the same semantic label to be close while the different ones to be far away.
• Figure 5: The illustrations for our patch jitter augmentation, patch shuffle and random shift. Color frames indicate the changed regions.