Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Freq-DP Net: A Dual-Branch Network for Fence Removal using Dual-Pixel and Fourier Priors

Kunal Swami, Sudha Velusamy, Chandra Sekhar Seelamantula

TL;DR

Freq-DP Net is proposed, a novel dual-branch network that fuses two complementary priors: a geometric prior from defocus disparity, modeled using an explicit cost volume, and a structural prior of the fence's global pattern, learned via Fast Fourier Convolution (FFC).

Abstract

Removing fence occlusions from single images is a challenging task that degrades visual quality and limits downstream computer vision applications. Existing methods often fail on static scenes or require motion cues from multiple frames. To overcome these limitations, we introduce the first framework to leverage dual-pixel (DP) sensors for this problem. We propose Freq-DP Net, a novel dual-branch network that fuses two complementary priors: a geometric prior from defocus disparity, modeled using an explicit cost volume, and a structural prior of the fence's global pattern, learned via Fast Fourier Convolution (FFC). An attention mechanism intelligently merges these cues for highly accurate fence segmentation. To validate our approach, we build and release a diverse benchmark with different fence varieties. Experiments demonstrate that our method significantly outperforms strong general-purpose baselines, establishing a new state-of-the-art for single-image, DP-based fence removal.

Freq-DP Net: A Dual-Branch Network for Fence Removal using Dual-Pixel and Fourier Priors

TL;DR

Freq-DP Net is proposed, a novel dual-branch network that fuses two complementary priors: a geometric prior from defocus disparity, modeled using an explicit cost volume, and a structural prior of the fence's global pattern, learned via Fast Fourier Convolution (FFC).

Abstract

Removing fence occlusions from single images is a challenging task that degrades visual quality and limits downstream computer vision applications. Existing methods often fail on static scenes or require motion cues from multiple frames. To overcome these limitations, we introduce the first framework to leverage dual-pixel (DP) sensors for this problem. We propose Freq-DP Net, a novel dual-branch network that fuses two complementary priors: a geometric prior from defocus disparity, modeled using an explicit cost volume, and a structural prior of the fence's global pattern, learned via Fast Fourier Convolution (FFC). An attention mechanism intelligently merges these cues for highly accurate fence segmentation. To validate our approach, we build and release a diverse benchmark with different fence varieties. Experiments demonstrate that our method significantly outperforms strong general-purpose baselines, establishing a new state-of-the-art for single-image, DP-based fence removal.
Paper Structure (17 sections, 3 equations, 8 figures, 3 tables, 1 algorithm)

This paper contains 17 sections, 3 equations, 8 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. DP Sensor
  4. Fence Removal
  5. Dataset Generation
  6. Real-World Dataset Collection
  7. Synthetic Dataset Generation
  8. Proposed Method
  9. Freq-DP Net
  10. DP Disparity Branch
  11. Structural Frequency Branch & Fusion
  12. Fence Removal
  13. Results and Discussion
  14. Fence Segmentation
  15. Fence Removal
  16. ...and 2 more sections

Figures (8)

  • Figure 1: From a single input (a), we leverage disparity between the left (b) and right (c) DP views to generate a precise mask (d), which guides a model to produce the final result (f). Zoomed-in patches (e) highlight the disparity cue. DP views are grayscale (from the sensor's green channel). We use 'left/right' terminology for consistency, the smartphone disparity is typically vertical. (g) shows our result on an out-of-distribution fence sample. Please zoom in for better detail.
  • Figure 2: DP sensor image formation.
  • Figure 3: Our dataset features a wide range of fence structures, varying in pattern, material, and thickness.
  • Figure 4: Real-world dataset capture setup, and real-world and synthetic dataset samples. Please zoom in for better detail.
  • Figure 5: Network architecture of Freq-DP Net.
  • ...and 3 more figures