EGSA-PT:Edge-Guided Spatial Attention with Progressive Training for Monocular Depth Estimation and Segmentation of Transparent Objects

TL;DR

The paper addresses the challenge of monocular depth estimation and semantic segmentation for transparent objects, where cross-task interference and boundary ambiguity hinder performance. It introduces Edge-Guided Spatial Attention (EGSA), which replaces channel attention with boundary-aware spatial gating guided by edge maps, and a Progressive Edge-Guided Training strategy that transitions from RGB-derived edges to depth-derived edges during training, along with iterative refinement and a joint loss for depth and segmentation. The authors show consistent depth improvements on Syn-TODD and ClearPose, with strongest gains in transparent regions, and demonstrate robustness to backbone changes by evaluating with DINOv2; the approach does not require ground-truth depth during training. Overall, EGSA and the progressive edge training strategy provide a boundary-aware fusion mechanism that mitigates cross-task bias in transparent-object perception and generalizes across datasets and backbones, offering practical benefits for robotics and vision systems dealing with transparency.

Abstract

Transparent object perception remains a major challenge in computer vision research, as transparency confounds both depth estimation and semantic segmentation. Recent work has explored multi-task learning frameworks to improve robustness, yet negative cross-task interactions often hinder performance. In this work, we introduce Edge-Guided Spatial Attention (EGSA), a fusion mechanism designed to mitigate destructive interactions by incorporating boundary information into the fusion between semantic and geometric features. On both Syn-TODD and ClearPose benchmarks, EGSA consistently improved depth accuracy over the current state of the art method (MODEST), while preserving competitive segmentation performance, with the largest improvements appearing in transparent regions. Besides our fusion design, our second contribution is a multi-modal progressive training strategy, where learning transitions from edges derived from RGB images to edges derived from predicted depth images. This approach allows the system to bootstrap learning from the rich textures contained in RGB images, and then switch to more relevant geometric content in depth maps, while it eliminates the need for ground-truth depth at training time. Together, these contributions highlight edge-guided fusion as a robust approach capable of improving transparent object perception.

Figures (4)

• Figure 1: EGSA-PT predicts accurate depth and segmentation masks in the presence of transparent materials.
• Figure 2: Overview of our proposed model architecture. A single RGB image is first encoded into multi-scale features with the encoder backbone. In parallel, multi-scale edges are injected into the EGSA block, together with multiscale features belonging to the depth and segmentation branch for fusion. The fused representations are then passed to the decoder for final depth and segmentation predictions.
• Figure 3: EGSA Block.
• Figure 4: Qualitative comparison on Syn-TODD. Rectangles highlight transparent regions where our method reduces depth error compared to MODEST. Unlike MODEST, which often over-smooths depth, our edge-guided fusion preserves sharper transitions. This demonstrates the utility of boundary-aware fusion in precisely the regions where transparent perception is challenging.