Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Application of Sensitivity Analysis Methods for Studying Neural Network Models

Jiaxuan Miao, Sergey Matveev

TL;DR

The paper addresses interpretability of neural networks by combining Sobol global sensitivity analysis with local sensitivity and activation maximization on both tabular and image datasets. It demonstrates that for a small diabetes-focused FFN, Sobol indices $S_i$ and $S_{T_i}$ can identify the most influential inputs, enabling effective feature reduction with minimal loss in accuracy. For CNNs, where global Sobol is intractable, it employs local pixel perturbation heatmaps and activation maximization, revealing how VGG-16 and ResNet-18 encode features and how AM can visualize class-specific patterns, including medical imaging features in ultrasound data and comparisons to Grad-CAM. The findings support improved feature selection, robustness assessment, and medical image interpretation, while also outlining limitations in scaling Sobol to large CNNs and in ensuring AM outputs align with real-world features.

Abstract

This study demonstrates the capabilities of several methods for analyzing the sensitivity of neural networks to perturbations of the input data and interpreting their underlying mechanisms. The investigated approaches include the Sobol global sensitivity analysis, the local sensitivity method for input pixel perturbations and the activation maximization technique. As examples, in this study we consider a small feedforward neural network for analyzing an open tabular dataset of clinical diabetes data, as well as two classical convolutional architectures, VGG-16 and ResNet-18, which are widely used in image processing and classification. Utilization of the global sensitivity analysis allows us to identify the leading input parameters of the chosen tiny neural network and reduce their number without significant loss of the accuracy. As far as global sensitivity analysis is not applicable to larger models we try the local sensitivity analysis and activation maximization method in application to the convolutional neural networks. These methods show interesting patterns for the convolutional models solving the image classification problem. All in all, we compare the results of the activation maximization method with popular Grad-CAM technique in the context of ultrasound data analysis.

Application of Sensitivity Analysis Methods for Studying Neural Network Models

TL;DR

The paper addresses interpretability of neural networks by combining Sobol global sensitivity analysis with local sensitivity and activation maximization on both tabular and image datasets. It demonstrates that for a small diabetes-focused FFN, Sobol indices and can identify the most influential inputs, enabling effective feature reduction with minimal loss in accuracy. For CNNs, where global Sobol is intractable, it employs local pixel perturbation heatmaps and activation maximization, revealing how VGG-16 and ResNet-18 encode features and how AM can visualize class-specific patterns, including medical imaging features in ultrasound data and comparisons to Grad-CAM. The findings support improved feature selection, robustness assessment, and medical image interpretation, while also outlining limitations in scaling Sobol to large CNNs and in ensuring AM outputs align with real-world features.

Abstract

This study demonstrates the capabilities of several methods for analyzing the sensitivity of neural networks to perturbations of the input data and interpreting their underlying mechanisms. The investigated approaches include the Sobol global sensitivity analysis, the local sensitivity method for input pixel perturbations and the activation maximization technique. As examples, in this study we consider a small feedforward neural network for analyzing an open tabular dataset of clinical diabetes data, as well as two classical convolutional architectures, VGG-16 and ResNet-18, which are widely used in image processing and classification. Utilization of the global sensitivity analysis allows us to identify the leading input parameters of the chosen tiny neural network and reduce their number without significant loss of the accuracy. As far as global sensitivity analysis is not applicable to larger models we try the local sensitivity analysis and activation maximization method in application to the convolutional neural networks. These methods show interesting patterns for the convolutional models solving the image classification problem. All in all, we compare the results of the activation maximization method with popular Grad-CAM technique in the context of ultrasound data analysis.

Paper Structure

This paper contains 13 sections, 17 equations, 16 figures, 1 table.

Table of Contents

  1. Introduction
  2. Global Sensitivity Analysis of a Feedforward Neural Network
  3. Introduction to Data and Model
  4. Sobol Method for Global Sensitivity Analysis
  5. Result for Tiny Neural Network
  6. Comparison with PCA
  7. Local sensitivity analysis for convolutional neural networks
  8. Numerical Results
  9. Activation maximization method
  10. Application of AM to VGG-16
  11. Sensitivity analysis for ResNet-18
  12. Application of VGG-16 and activation maximization to medical data
  13. Conclusion

Figures (16)

  • Figure 1: The Sobol indices $S_1$ and $S_T$ for the output.
  • Figure 2: The second-order Sobol indices $S_2$ for the output.
  • Figure 3: $S_1$ for the first hidden layer.
  • Figure 4: $S_1$ and $S_T$ for various physiological indicators and sample sizes $N$. The shaded areas represent the corresponding confidence intervals.
  • Figure 5: The sensitivity heatmap for the three color channels of block 1. The scale is $10^{0}$.
  • ...and 11 more figures