Internal-External Boundary Attention Fusion for Glass Surface Segmentation

TL;DR

The paper tackles glass surface segmentation from single-color images, where reflections and transmitted backgrounds blur boundaries and hinder detection. It introduces two boundary-focused modules, IEBAM and FBAM, built on a DeeplabV3+-based backbone, along with a decomposed contour loss to guide precise boundary localization. IEBAM separates and learns internal and external boundary cues plus body features, while FBAM fuses these cues with semantic guidance to reinforce glass regions. Across six public datasets, the approach achieves state-of-the-art results, with ablations confirming the contributions and highlighting the strong role of external boundary information for robust generalization.

Abstract

Glass surfaces of transparent objects and mirrors are not able to be uniquely and explicitly characterized by their visual appearances because they contain the visual appearance of other reflected or transmitted surfaces as well. Detecting glass regions from a single-color image is a challenging task. Recent deep-learning approaches have paid attention to the description of glass surface boundary where the transition of visual appearances between glass and non-glass surfaces are observed. In this work, we analytically investigate how glass surface boundary helps to characterize glass objects. Inspired by prior semantic segmentation approaches with challenging image types such as X-ray or CT scans, we propose separated internal-external boundary attention modules that individually learn and selectively integrate visual characteristics of the inside and outside region of glass surface from a single color image. Our proposed method is evaluated on six public benchmarks comparing with state-of-the-art methods showing promising results.

Figures (7)

• Figure 1: Glass objects: Glass cup with strong internal but weak external boundaries and Window with strong external boundary
• Figure 2: The pipeline of our proposed network. It can be observed that with the ASPP structure, our baseline is DeeplabV3+, and we connect our IEBAM and FBAM in a cascaded order.
• Figure 3: (a) IEBAM: ${\textit{L}}_{ex}$, ${\textit{L}}_{in}$, ${\textit{L}}_{b}$, ${\textit{L}}_{body}$ are loss functions (b) Vislualized Residual Features: Second and third images are PCA visualizations of residual features with entire boundary and internal boundary features, respectively. Using only internal boundary reduces noise compared to entire boundary.
• Figure 4: The structure of our proposed FBAM. ${\textit{L}}_{m}$ is the loss of entire glass prediction.
• Figure 5: (a) Ground truth of a glass surface object (b) Internal boundary, external boundary, real boundary, body and merged regions of glass object with ground truth distribution