Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Compressive Feature Selection for Remote Visual Multi-Task Inference

Saeed Ranjbar Alvar, Ivan V. Bajić

TL;DR

This paper addresses feature selection for multi-task DNNs in edge-to-cloud inference by introducing a task-aware mutual information (MI) framework. It develops a tractable MI estimator that operates on patch-level features and clustered output patches, enabling per-task feature importance scores $I_{i,j}=I(\widetilde{X}_i;\widehat{Y}_j)$ and supporting both hard and soft feature selection. Empirically, MI-based selection outperforms norm-based baselines, particularly for reconstruction tasks, and soft selection reduces bitrate while preserving accuracy, aided by HEVC compression of enhancement features. The method is extended with a multi-objective distortion analysis to account for varying task priorities, showing MI-based selection is favorable across a broad region of the task-weight space, with practical implications for efficient remote inference on edge devices.

Abstract

Deep models produce a number of features in each internal layer. A key problem in applications such as feature compression for remote inference is determining how important each feature is for the task(s) performed by the model. The problem is especially challenging in the case of multi-task inference, where the same feature may carry different importance for different tasks. In this paper, we examine how effective is mutual information (MI) between a feature and a model's task output as a measure of the feature's importance for that task. Experiments involving hard selection and soft selection (unequal compression) based on MI are carried out to compare the MI-based method with alternative approaches. Multi-objective analysis is provided to offer further insight.

Compressive Feature Selection for Remote Visual Multi-Task Inference

TL;DR

This paper addresses feature selection for multi-task DNNs in edge-to-cloud inference by introducing a task-aware mutual information (MI) framework. It develops a tractable MI estimator that operates on patch-level features and clustered output patches, enabling per-task feature importance scores and supporting both hard and soft feature selection. Empirically, MI-based selection outperforms norm-based baselines, particularly for reconstruction tasks, and soft selection reduces bitrate while preserving accuracy, aided by HEVC compression of enhancement features. The method is extended with a multi-objective distortion analysis to account for varying task priorities, showing MI-based selection is favorable across a broad region of the task-weight space, with practical implications for efficient remote inference on edge devices.

Abstract

Deep models produce a number of features in each internal layer. A key problem in applications such as feature compression for remote inference is determining how important each feature is for the task(s) performed by the model. The problem is especially challenging in the case of multi-task inference, where the same feature may carry different importance for different tasks. In this paper, we examine how effective is mutual information (MI) between a feature and a model's task output as a measure of the feature's importance for that task. Experiments involving hard selection and soft selection (unequal compression) based on MI are carried out to compare the MI-based method with alternative approaches. Multi-objective analysis is provided to offer further insight.
Paper Structure (7 sections, 9 equations, 8 figures)

This paper contains 7 sections, 9 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Related work
  3. Proposed methods
  4. Estimating MI in visual multi-task DNNs
  5. Feature selection
  6. Experiments
  7. Conclusions

Figures (8)

  • Figure 1: Mutual information between an $N\times N$ feature patch $\widetilde{X}$ and the corresponding $M\times M$ output patch $\widetilde{Y}$
  • Figure 2: True and estimated MI for 1D Gaussian random variables (top) and 2D Gaussian random vectors (bottom).
  • Figure 3: Illustration of soft feature selection.
  • Figure 4: Three-task DNN from Saeed_ICASSP_2020 used in our experiments. Heads 1-3 are similar to FC8 networks FC.
  • Figure 5: Task accuracy vs. hard selection % for semantic segmentation (left), disparity map estimation (middle) and input reconstruction (right)
  • ...and 3 more figures