Just Noticeable Difference Modeling for Deep Visual Features

TL;DR

FeatJND introduces a task-aligned just noticeable difference for deep visual features by learning a per-feature perturbation map $\delta$ that maximizes allowable distortion while preserving downstream performance within a tolerance $\varepsilon$. A lightweight estimator $G_{\theta}$ learns $\delta$ from feature inputs at designated interface points, balancing distortion magnitude $\mathcal{M}(\delta)$ with a differentiable downstream discrepancy $D_t$ across classification, detection, and segmentation. The method is validated on ImageNet and COCO, showing FeatJND distortions outperform Gaussian noise at matched distortion levels and enabling improved token-wise quantization via JND-guided step sizes. This provides a practical, task-aware criterion to manage feature quality under bandwidth and compute constraints in feature-based vision systems.

Abstract

Deep visual features are increasingly used as the interface in vision systems, motivating the need to describe feature characteristics and control feature quality for machine perception. Just noticeable difference (JND) characterizes the maximum imperceptible distortion for images under human or machine vision. Extending it to deep visual features naturally meets the above demand by providing a task-aligned tolerance boundary in feature space, offering a practical reference for controlling feature quality under constrained resources. We propose FeatJND, a task-aligned JND formulation that predicts the maximum tolerable per-feature perturbation map while preserving downstream task performance. We propose a FeatJND estimator at standardized split points and validate it across image classification, detection, and instance segmentation. Under matched distortion strength, FeatJND-based distortions consistently preserve higher task performance than unstructured Gaussian perturbations, and attribution visualizations suggest FeatJND can suppress non-critical feature regions. As an application, we further apply FeatJND to token-wise dynamic quantization and show that FeatJND-guided step-size allocation yields clear gains over random step-size permutation and global uniform step size under the same noise budget. Our code will be released after publication.

Figures (19)

• Figure 1: Illustration of the just noticeable difference for deep visual features (FeatJND). In this paper, we first construct the concept of FeatJND, and then evaluate it on different tasks, and verify its effectiveness on a token-wise quantization application, which means spatially-varying step sizes shared across channels.
• Figure 2: The split point selection of the downstream-task network.
• Figure 3: Training scheme and architecture of the JND estimator. BN means batch normalization.
• Figure 4: Task performance comparison for image classification between FeatJND-based and Gaussian noise distortions.
• Figure 5: Task performance comparison for object detection between FeatJND-based and Gaussian noise distortions.