Structural and Statistical Texture Knowledge Distillation and Learning for Segmentation

TL;DR

This work introduces Structural and Statistical Texture Knowledge Distillation (SSTKD) to address the lack of low-level texture guidance in semantic segmentation. It pairs a structural pathway (Contourlet Decomposition Module) with a statistical pathway (Texture Intensity Equalization Module) to distill texture knowledge from a teacher to a student, supervised by dedicated losses and integrated into STLNet++ and U-SSNet. Ablations demonstrate that both texture types contribute and interact synergistically, yielding state-of-the-art results on Cityscapes, ADE20K, and Pascal VOC 2012, and strong performance on ultra-high-resolution segmentation. The proposed framework generalizes beyond segmentation to dense prediction tasks like object detection, offering a practical approach to incorporate low-level texture into modern deep learning models.

Abstract

Low-level texture feature/knowledge is also of vital importance for characterizing the local structural pattern and global statistical properties, such as boundary, smoothness, regularity, and color contrast, which may not be well addressed by high-level deep features. In this paper, we aim to re-emphasize the low-level texture information in deep networks for semantic segmentation and related knowledge distillation tasks. To this end, we take full advantage of both structural and statistical texture knowledge and propose a novel Structural and Statistical Texture Knowledge Distillation (SSTKD) framework for semantic segmentation. Specifically, Contourlet Decomposition Module (CDM) is introduced to decompose the low-level features with iterative Laplacian pyramid and directional filter bank to mine the structural texture knowledge, and Texture Intensity Equalization Module (TIEM) is designed to extract and enhance the statistical texture knowledge with the corresponding Quantization Congruence Loss (QDL). Moreover, we propose the Co-occurrence TIEM (C-TIEM) and generic segmentation frameworks, namely STLNet++ and U-SSNet, to enable existing segmentation networks to harvest the structural and statistical texture information more effectively. Extensive experimental results on three segmentation tasks demonstrate the effectiveness of the proposed methods and their state-of-the-art performance on seven popular benchmark datasets, respectively.

Figures (15)

• Figure 1: An example of the structural and statistical texture knowledge distillation. The original texture is fuzzy and in low-contrast. After distillation, the contour is clearer and the intensity contrast is more equalized, indicating a clear enhancement of both texture types.
• Figure 2: The overview of SSTKD. Apart from the response knowledge, we propose to extract the texture knowledge from low-level features. The corresponding parts of two kinds of texture knowledge are presented in Figs. \ref{['cdm']}, \ref{['lp']} and \ref{['fig:tiem']}, respectively.
• Figure 3: Details of CDM do2005contourletdonoho2001canc-cnn.
• Figure 4: Details of LP decomposition do2005contourletdonoho2001canc-cnn.
• Figure 5: (a) The wavelet possesses square supports, mainly suitable for capturing point discontinuities. (b) The contourlet is able to capture linear segments of contours with fewer coefficients do2005contourletdonoho2001canc-cnn.