Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning

Dongkwan Lee, Junhoo Lee, Nojun Kwak

TL;DR

The paper addresses model vulnerability to perturbations and domain shifts by positing that deep features encode semantic, task-relevant signals predominantly in high-frequency components while domain-specific biases reside in low-frequency components. It introduces Deep Edge Filter, a high-pass filtering operation applied to deep features (h_edge = h - LPF(h)) that is attached at a single layer and is non-trainable, with architecture-specific LPF choices (2D for CNNs, 1D for Transformers/MLPs). Through extensive experiments across Vision (TTA), Language (GLUE SST-2 and related tasks), 3D NeRF, and Audio (UrbanSound8K), the method yields consistent generalization and robustness gains, accompanied by evidence of induced sparsity and favorable frequency decomposition. Ablation studies confirm the importance of filter positioning and LPF type, while comparisons against trainable convolution layers suggest that the edge-filtered approach provides a genuine generalization boost beyond merely increasing compute. The work suggests broad applicability, including potential extensions to foundation models and multimodal systems, and provides open-source code for replication.

Abstract

We introduce the Deep Edge Filter, a novel approach that applies high-pass filtering to deep neural network features to improve model generalizability. Our method is motivated by our hypothesis that neural networks encode task-relevant semantic information in high-frequency components while storing domain-specific biases in low-frequency components of deep features. By subtracting low-pass filtered outputs from original features, our approach isolates generalizable representations while preserving architectural integrity. Experimental results across diverse domains such as Vision, Text, 3D, and Audio demonstrate consistent performance improvements regardless of model architecture and data modality. Analysis reveals that our method induces feature sparsification and effectively isolates high-frequency components, providing empirical validation of our core hypothesis. The code is available at https://github.com/dongkwani/DeepEdgeFilter.

Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning

TL;DR

The paper addresses model vulnerability to perturbations and domain shifts by positing that deep features encode semantic, task-relevant signals predominantly in high-frequency components while domain-specific biases reside in low-frequency components. It introduces Deep Edge Filter, a high-pass filtering operation applied to deep features (h_edge = h - LPF(h)) that is attached at a single layer and is non-trainable, with architecture-specific LPF choices (2D for CNNs, 1D for Transformers/MLPs). Through extensive experiments across Vision (TTA), Language (GLUE SST-2 and related tasks), 3D NeRF, and Audio (UrbanSound8K), the method yields consistent generalization and robustness gains, accompanied by evidence of induced sparsity and favorable frequency decomposition. Ablation studies confirm the importance of filter positioning and LPF type, while comparisons against trainable convolution layers suggest that the edge-filtered approach provides a genuine generalization boost beyond merely increasing compute. The work suggests broad applicability, including potential extensions to foundation models and multimodal systems, and provides open-source code for replication.

Abstract

We introduce the Deep Edge Filter, a novel approach that applies high-pass filtering to deep neural network features to improve model generalizability. Our method is motivated by our hypothesis that neural networks encode task-relevant semantic information in high-frequency components while storing domain-specific biases in low-frequency components of deep features. By subtracting low-pass filtered outputs from original features, our approach isolates generalizable representations while preserving architectural integrity. Experimental results across diverse domains such as Vision, Text, 3D, and Audio demonstrate consistent performance improvements regardless of model architecture and data modality. Analysis reveals that our method induces feature sparsification and effectively isolates high-frequency components, providing empirical validation of our core hypothesis. The code is available at https://github.com/dongkwani/DeepEdgeFilter.
Paper Structure (32 sections, 4 equations, 7 figures, 10 tables)

This paper contains 32 sections, 4 equations, 7 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Frequency Perspective in Deep Learning
  4. Activation Filtering and Sparsity
  5. Method: Deep Edge Filters
  6. Rethinking the Role of Edge Filters
  7. Linear Probing Observation.
  8. Decomposition of Deep Features.
  9. Empirical Justification from Domain Adaptation.
  10. Sparse Coding through High-Pass Filtering.
  11. Deep Edge Filter
  12. Experiments
  13. Modality Selection.
  14. Overview.
  15. Vision: Test-Time Adaptation
  16. ...and 17 more sections

Figures (7)

  • Figure 1: Example of a model architecture with Edge Filter attached. This figure illustrates the case when the Edge Filter is attached after Block 2 of the base model.
  • Figure 2: Qualitative analysis on NeRF. Overall, Edge filtering reduces floating artifacts.
  • Figure 3: Statistical analysis of the impact of Edge Filter on deep features. \ref{['subfig:nofilter']} and \ref{['subfig:edgefilter']} show the sparsity of output features for each block when training a vanilla model and a WRN-28-10 model with Edge Filter attached to block1 on the CIFAR-10 dataset, respectively. \ref{['subfig:fft']} represents the FFT amplitude spectra of both inputs and outputs from the Edge Filter when evaluated on the validation set after the training in \ref{['subfig:edgefilter']} is completed.
  • Figure 4: Heatmap of performance gain (%p) compared to the cases without Edge Filter, depending on the position and the kernel size of the Edge Filter. CIFAR-10C TTA benchmark is used in the vision domain, SST-2 in the language domain, and UrbanSound8k classification in the audio domain.
  • Figure 5: Heatmap of performance gains as multiples of standard deviation ($\mathbf{\sigma}$), represents the statistical significance of \ref{['fig:heatmap']}.
  • ...and 2 more figures