Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SERNet-Former: Semantic Segmentation by Efficient Residual Network with Attention-Boosting Gates and Attention-Fusion Networks

Serdar Erisen

TL;DR

<3-5 sentence high-level summary> SERNet-Former tackles the efficiency-accuracy trade-off in semantic segmentation by integrating an Efficient-ResNet encoder with attention-boosting gates/modules and an attention-fusion decoder. A dilation-based bridge (DbN) and skip connections further refine multi-scale context fusion, enabling compact models to achieve strong mean IoU on CamVid and Cityscapes with fewer parameters than many baselines. Ablation studies quantify the contributions of AbG/AbM/AfN/DbN, illustrating substantial performance gains from attention-driven fusion in both encoder and decoder. The work suggests promising directions for extending to RGB-D, 3D contexts, and real-time deployment under limited hardware resources.

Abstract

Improving the efficiency of state-of-the-art methods in semantic segmentation requires overcoming the increasing computational cost as well as issues such as fusing semantic information from global and local contexts. Based on the recent success and problems that convolutional neural networks (CNNs) encounter in semantic segmentation, this research proposes an encoder-decoder architecture with a unique efficient residual network, Efficient-ResNet. Attention-boosting gates (AbGs) and attention-boosting modules (AbMs) are deployed by aiming to fuse the equivariant and feature-based semantic information with the equivalent sizes of the output of global context of the efficient residual network in the encoder. Respectively, the decoder network is developed with the additional attention-fusion networks (AfNs) inspired by AbM. AfNs are designed to improve the efficiency in the one-to-one conversion of the semantic information by deploying additional convolution layers in the decoder part. Our network is tested on the challenging CamVid and Cityscapes datasets, and the proposed methods reveal significant improvements on the residual networks. To the best of our knowledge, the developed network, SERNet-Former, achieves state-of-the-art results (84.62 % mean IoU) on CamVid dataset and challenging results (87.35 % mean IoU) on Cityscapes validation dataset.

SERNet-Former: Semantic Segmentation by Efficient Residual Network with Attention-Boosting Gates and Attention-Fusion Networks

TL;DR

<3-5 sentence high-level summary> SERNet-Former tackles the efficiency-accuracy trade-off in semantic segmentation by integrating an Efficient-ResNet encoder with attention-boosting gates/modules and an attention-fusion decoder. A dilation-based bridge (DbN) and skip connections further refine multi-scale context fusion, enabling compact models to achieve strong mean IoU on CamVid and Cityscapes with fewer parameters than many baselines. Ablation studies quantify the contributions of AbG/AbM/AfN/DbN, illustrating substantial performance gains from attention-driven fusion in both encoder and decoder. The work suggests promising directions for extending to RGB-D, 3D contexts, and real-time deployment under limited hardware resources.

Abstract

Improving the efficiency of state-of-the-art methods in semantic segmentation requires overcoming the increasing computational cost as well as issues such as fusing semantic information from global and local contexts. Based on the recent success and problems that convolutional neural networks (CNNs) encounter in semantic segmentation, this research proposes an encoder-decoder architecture with a unique efficient residual network, Efficient-ResNet. Attention-boosting gates (AbGs) and attention-boosting modules (AbMs) are deployed by aiming to fuse the equivariant and feature-based semantic information with the equivalent sizes of the output of global context of the efficient residual network in the encoder. Respectively, the decoder network is developed with the additional attention-fusion networks (AfNs) inspired by AbM. AfNs are designed to improve the efficiency in the one-to-one conversion of the semantic information by deploying additional convolution layers in the decoder part. Our network is tested on the challenging CamVid and Cityscapes datasets, and the proposed methods reveal significant improvements on the residual networks. To the best of our knowledge, the developed network, SERNet-Former, achieves state-of-the-art results (84.62 % mean IoU) on CamVid dataset and challenging results (87.35 % mean IoU) on Cityscapes validation dataset.
Paper Structure (16 sections, 3 equations, 3 figures, 5 tables)

This paper contains 16 sections, 3 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Method
  4. Efficient-ResNet: Encoder with attention-boosting modules
  5. Dilation-based separable convolution network
  6. Decoder with attention-fusion networks
  7. Loss function, the classification layer and the evaluation metrics
  8. Evaluation metrics
  9. Experiments and Results
  10. Datasets
  11. The Cambridge-driving Labelled Video Database (CamVid)
  12. Cityscapes
  13. Implementation details and experiment results
  14. Comparison with regard to state-of-the-art
  15. Ablation studies
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Training progress of SERNet-Former and the selected baselines of DeepLabv3+ on CamVid dataset.
  • Figure 2: Schematic illustration of SERNet-Former. (a) Attention-boosting Gate (AbG) and Attention-boosting Module (AbM) are fused into the encoder part. (b) Attention-fusion Network (AfN), introduced into the decoder
  • Figure 3: Examples from the test results on Cityscapes validation dataset.