Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Single Image Reflection Separation via Dual Prior Interaction Transformer

Yue Huang, Zi'ang Li, Tianle Hu, Jie Wen, Guanbin Li, Jinglin Zhang, Guoxu Zhou, Xiaozhao Fang

TL;DR

This work tackles single-image reflection separation by introducing a Dual-Prior Interaction Transformer (DPIT) that jointly leverages a lightweight transmission-prior generator and a dual-prior fusion mechanism. The Local Linear Correction Network (LLCN) converts the problem into adaptive linear correction via the physical model $T = S I + B$, producing high-quality priors with few parameters, while the Dual-Prior Feature Interactive Network uses Dual-Stream Channel Reorganization Attention to fuse general priors from a Swin Transformer with transmission priors for deep, complementary fusion. The approach achieves state-of-the-art performance on five real-world datasets with competitive efficiency, as evidenced by an average PSNR of 27.21 dB and SSIM of 0.924, along with detailed ablations validating the contributions of LLCN and DSCRAB. By separating the roles of prior generation and prior fusion, DPIT provides a robust, scalable framework for reflection removal and has potential applications in other low-level restoration tasks that require multi-prior collaboration.

Abstract

Single image reflection separation aims to separate the transmission and reflection layers from a mixed image. Existing methods typically combine general priors from pre-trained models with task-specific priors such as text prompts and reflection detection. However, the transmission prior, as the most direct task-specific prior for the target transmission layer, has not been effectively modeled or fully utilized, limiting performance in complex scenarios. To address this issue, we propose a dual-prior interaction framework based on lightweight transmission prior generation and effective prior fusion. First, we design a Local Linear Correction Network (LLCN) that finetunes pre-trained models based on the physical constraint T=SI+B, where S and B represent pixel-wise and channel-wise scaling and bias transformations. LLCN efficiently generates high-quality transmission priors with minimal parameters. Second, we construct a Dual-Prior Interaction Transformer (DPIT) that employs a dual-stream channel reorganization attention mechanism. By reorganizing features from general and transmission priors for attention computation, DPIT achieves deep fusion of both priors, fully exploiting their complementary information. Experimental results on multiple benchmark datasets demonstrate that the proposed method achieves state-of-the-art performance.

Single Image Reflection Separation via Dual Prior Interaction Transformer

TL;DR

This work tackles single-image reflection separation by introducing a Dual-Prior Interaction Transformer (DPIT) that jointly leverages a lightweight transmission-prior generator and a dual-prior fusion mechanism. The Local Linear Correction Network (LLCN) converts the problem into adaptive linear correction via the physical model , producing high-quality priors with few parameters, while the Dual-Prior Feature Interactive Network uses Dual-Stream Channel Reorganization Attention to fuse general priors from a Swin Transformer with transmission priors for deep, complementary fusion. The approach achieves state-of-the-art performance on five real-world datasets with competitive efficiency, as evidenced by an average PSNR of 27.21 dB and SSIM of 0.924, along with detailed ablations validating the contributions of LLCN and DSCRAB. By separating the roles of prior generation and prior fusion, DPIT provides a robust, scalable framework for reflection removal and has potential applications in other low-level restoration tasks that require multi-prior collaboration.

Abstract

Single image reflection separation aims to separate the transmission and reflection layers from a mixed image. Existing methods typically combine general priors from pre-trained models with task-specific priors such as text prompts and reflection detection. However, the transmission prior, as the most direct task-specific prior for the target transmission layer, has not been effectively modeled or fully utilized, limiting performance in complex scenarios. To address this issue, we propose a dual-prior interaction framework based on lightweight transmission prior generation and effective prior fusion. First, we design a Local Linear Correction Network (LLCN) that finetunes pre-trained models based on the physical constraint T=SI+B, where S and B represent pixel-wise and channel-wise scaling and bias transformations. LLCN efficiently generates high-quality transmission priors with minimal parameters. Second, we construct a Dual-Prior Interaction Transformer (DPIT) that employs a dual-stream channel reorganization attention mechanism. By reorganizing features from general and transmission priors for attention computation, DPIT achieves deep fusion of both priors, fully exploiting their complementary information. Experimental results on multiple benchmark datasets demonstrate that the proposed method achieves state-of-the-art performance.
Paper Structure (20 sections, 27 equations, 5 figures, 4 tables)

This paper contains 20 sections, 27 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. Prior Construction
  4. Physical Model Construction
  5. PROPOSED METHOD
  6. Overall Architecture
  7. Local Linear Correction Model
  8. Dual-Stream Channel Reorganization Attention
  9. Loss Function
  10. Experimental Validation
  11. Data Preparation
  12. Training Data
  13. Validation Data
  14. Test Data
  15. Performance Evaluation
  16. ...and 5 more sections

Figures (5)

  • Figure 1: Architecture of the Dual-Prior Interactive Transformer (DPIT), including General Prior Feature Extraction Network, Transmission Prior Feature Extraction Network (TPFEN), Local Linear Correction Network (LLCN), Dual-Prior Feature Extraction Network (DPFEN), and Local Linear Correction Network (LLCN).
  • Figure 2: Comparison of three dual-stream interaction modules, including MuGI Block, DAI Block, and DSCRA Block.
  • Figure 3: Comparison of single image reflection removal results by different methods on samples from Objects, Postcard, and Wild datasets
  • Figure 4: Comparison of single image reflection removal results by different methods on samples from Real20, Nature, and Reflection Removal in the Wild (RRW) datasets
  • Figure 5: Comparison of reflection or other non-transmission component separation results by different methods on the Real45 dataset