Derivative based global sensitivity analysis and its entropic link

TL;DR

The paper addresses the limitations of variance-based global sensitivity analysis for outputs with skewed or multimodal distributions by proposing a derivative-based upper bound for the total effect entropy, $H_{T_i} \le H(X_i) + l_i$, where $l_i = \mathbb{E}[\ln|\partial g/\partial x_i|]$. To avoid negative sensitivity values and improve interpretability, it introduces an exponential entropy proxy $\kappa_{T_i} = e^{H_{T_i}}/e^{H(Y)}$ and derives its bound, linking derivative information to entropy-based measures without requiring input independence. Numerical results show the bound is tight for monotonic functions, and the exponential proxy yields rankings similar to entropy- and variance-based indices across Ishigami, G-function, and a flood-model example, with substantial computational savings. The work provides a practical, scalable screening tool for entropy-based GSA, capable of handling diverse distributions and offering a complement to Sobol' indices in complex models. It also points to future work on dependent inputs and density-estimation-based approaches to extend applicability to higher dimensions.

Abstract

Variance-based Sobol' sensitivity is one of the most well-known measures in global sensitivity analysis (GSA). However, uncertainties with certain distributions, such as highly skewed distributions or those with a heavy tail, cannot be adequately characterised using the second central moment only. Entropy-based GSA can consider the entire probability density function, but its application has been limited because it is difficult to estimate. Here we present a novel derivative-based upper bound for conditional entropies, to efficiently rank uncertain variables and to work as a proxy for entropy-based total effect indices. To overcome the non-desirable issue of negativity for differential entropies as sensitivity indices, we discuss an exponentiation of the total effect entropy and its proxy. Numerical verifications demonstrate that the upper bound is tight for monotonic functions and it provides the same input variable ranking as the entropy-based indices for about three-quarters of the 1000 random functions tested. We found that the new entropy proxy performs similarly to the variance-based proxies for a river flood physics model with 8 inputs of different distributions, and these two proxies are equivalent in the special case of linear functions with Gaussian inputs. We expect the new entropy proxy to increase the variable screening power of derivative-based GSA and to complement Sobol'-indices proxy for a more diverse type of distributions.

Figures (11)

• Figure 1: An overview for the relationship between entropy and variance proxies where the entropy proxies developed in this paper are highlighted in the box with dash lines.
• Figure 2: Histograms of the output for monotonic examples 1 to 3 as described in Section 4 of the Main Text. The exponential entropy of the outputs are (a) $e^{H(Y)} =2.26$; (b) $e^{H(Y)} =0.65$; (c) $e^{H(Y)} = 3.54$, which describes the effective extent/support of the underlying distribution. In comparison, the standard deviations of the output are 0.57, 0.22 and 0.91 for the three examples respectively.
• Figure 3: Surface plots for the monotonic functions in examples 1 - 3
• Figure 4: Convergence of the numerical estimated total effect entropy for monotonic examples 1 - 3.
• Figure 5: Example plot for the general functions considered. a) Scatter plot for the Ishigami function, $y = \sin(x_1) + 7 \sin^{2} (x_2) + 0.1 x^{4}_3\sin(x_1)$, $x_i \sim \mathbb{U} (-\pi, \pi)$ for $i = 1, 2, 3$. ; b) Surface plots for the G-function (2 variable plot), $y = \prod_{i=1}^{3} (|4x_i - 2| + a_i )/(1+a_i)$, $x_i \sim \mathbb{U} (0, 1)$ for $i = 1, 2, 3$. In this case, $a_i = (i-2)/2$, for $i = 1, 2, 3$. A lower value of $a_i$ indicates a higher importance of the input variable $x_i$, i.e., $x_1$ is the most important, while $x_3$ is the least important in this case

Theorems & Definitions (2)

• Theorem
• proof