Depth-Synergized Mamba Meets Memory Experts for All-Day Image Reflection Separation

TL;DR

This work tackles the challenging task of image reflection separation, especially at night when transmission and reflection layers have similar contrasts. It introduces DMDNet, which combines Depth-Aware Scanning (DAScan), Depth-Synergized State-Space Model (DS-SSM), and Memory Expert Compensation Module (MECM) to decouple layers using depth-guided saliency and cross-image memory. A dedicated NightIRS dataset of 1,000 nighttime image triplets supports evaluation under challenging low-light conditions. Experimental results show state-of-the-art performance on both daytime and nighttime data, supported by extensive ablations and qualitative analyses. The approach offers a practical, hardware-free solution for robust all-day reflection separation with potential applications in surveillance, photography, and downstream vision tasks.

Abstract

Image reflection separation aims to disentangle the transmission layer and the reflection layer from a blended image. Existing methods rely on limited information from a single image, tending to confuse the two layers when their contrasts are similar, a challenge more severe at night. To address this issue, we propose the Depth-Memory Decoupling Network (DMDNet). It employs the Depth-Aware Scanning (DAScan) to guide Mamba toward salient structures, promoting information flow along semantic coherence to construct stable states. Working in synergy with DAScan, the Depth-Synergized State-Space Model (DS-SSM) modulates the sensitivity of state activations by depth, suppressing the spread of ambiguous features that interfere with layer disentanglement. Furthermore, we introduce the Memory Expert Compensation Module (MECM), leveraging cross-image historical knowledge to guide experts in providing layer-specific compensation. To address the lack of datasets for nighttime reflection separation, we construct the Nighttime Image Reflection Separation (NightIRS) dataset. Extensive experiments demonstrate that DMDNet outperforms state-of-the-art methods in both daytime and nighttime.

Figures (14)

• Figure 1: Proximity maps obtained by depth estimation across daytime, nighttime, and indoor scenes. Depth estimation sees through reflection occlusion to capture the underlying structures of $\boldsymbol{T}$.
• Figure 2: Depth-Memory Decoupling Network (DMDNet). DMDNet employs the DMBlock to decouple $\boldsymbol{T}$ and $\boldsymbol{R}$ using depth and memory cues.
• Figure 3: DMBlock and DSMamba. DSMamba prioritizes salient structures via DAScan and synergistically modulates state activations through DS-SSM. The numbers indicate the forward scanning order.
• Figure 4: Memory Expert Compensation Module (MECM) and its components, which leverage cross-image historical knowledge to guide the decoupling. Each memory expert consists of the GPStream for global adjustment and memory evolution, and the SCStream for spatial-level refinement.
• Figure 5: Examples from the NightIRS dataset. $\boldsymbol{I}$, $\boldsymbol{T}$, and $\boldsymbol{R}$ denote the blended image, transmission layer, and reflection layer, respectively.