Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Measuring Feature Dependency of Neural Networks by Collapsing Feature Dimensions in the Data Manifold

Yinzhu Jin, Matthew B. Dwyer, P. Thomas Fletcher

TL;DR

This paper introduces a new technique to measure the feature dependency of neural network models by moving data points along the feature dimension to a baseline feature value while staying on the data manifold, as estimated by a deep generative model.

Abstract

This paper introduces a new technique to measure the feature dependency of neural network models. The motivation is to better understand a model by querying whether it is using information from human-understandable features, e.g., anatomical shape, volume, or image texture. Our method is based on the principle that if a model is dependent on a feature, then removal of that feature should significantly harm its performance. A targeted feature is "removed" by collapsing the dimension in the data distribution that corresponds to that feature. We perform this by moving data points along the feature dimension to a baseline feature value while staying on the data manifold, as estimated by a deep generative model. Then we observe how the model's performance changes on the modified test data set, with the target feature dimension removed. We test our method on deep neural network models trained on synthetic image data with known ground truth, an Alzheimer's disease prediction task using MRI and hippocampus segmentations from the OASIS-3 dataset, and a cell nuclei classification task using the Lizard dataset.

Measuring Feature Dependency of Neural Networks by Collapsing Feature Dimensions in the Data Manifold

TL;DR

This paper introduces a new technique to measure the feature dependency of neural network models by moving data points along the feature dimension to a baseline feature value while staying on the data manifold, as estimated by a deep generative model.

Abstract

This paper introduces a new technique to measure the feature dependency of neural network models. The motivation is to better understand a model by querying whether it is using information from human-understandable features, e.g., anatomical shape, volume, or image texture. Our method is based on the principle that if a model is dependent on a feature, then removal of that feature should significantly harm its performance. A targeted feature is "removed" by collapsing the dimension in the data distribution that corresponds to that feature. We perform this by moving data points along the feature dimension to a baseline feature value while staying on the data manifold, as estimated by a deep generative model. Then we observe how the model's performance changes on the modified test data set, with the target feature dimension removed. We test our method on deep neural network models trained on synthetic image data with known ground truth, an Alzheimer's disease prediction task using MRI and hippocampus segmentations from the OASIS-3 dataset, and a cell nuclei classification task using the Lizard dataset.
Paper Structure (7 sections, 2 equations, 7 figures, 2 tables, 1 algorithm)

This paper contains 7 sections, 2 equations, 7 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methods
  3. Experiments
  4. Setup
  5. Results
  6. Compliance with Ethical Standards
  7. Acknowledgements

Figures (7)

  • Figure 1: Illustration of integrating a feature gradient in the ambient data space, where the resulting endpoint, $x'$ lands off of the data manifold (left). Example of this effect using aspect ratio of an ellipse as the feature (right).
  • Figure 1: Accuracies of evaluated classifiers on the original and reconstructed data. (mean $\pm$ std)
  • Figure 2: Illustration of the proposed feature collapsing method.
  • Figure 2: The results of our methods: accuracy after collapse (AAC) along each feature dimension with features of substantial performance drops (relative to RLF) shown in bold; and the CaCE scores with two different sets of percentiles to represent low and high feature values.
  • Figure 3: Sample cropped images from OASIS-3 (top) and Lizard (bottom).
  • ...and 2 more figures