Methods to integrate multinormals and compute classification measures

TL;DR

This work tackles the challenge of computing probabilities and classification performance for normal distributions in arbitrary dimensions and domains, where general analytical expressions are unavailable. It introduces two complementary approaches: a generalized chi-square method for quadratic domains and a ray-trace method for general, non-quadratic domains, together with open-source Matlab tools for integrating normals, obtaining distributions of functions of normals, and evaluating Bayes- and custom-classifier performance. The authors provide rigorous formulations for Bayes decision boundaries, the discriminability index $d'_b$, and dimension-wise contributions, and demonstrate the methods on vision-science problems such as occluding-target detection and camouflage, with performance benchmarks showing high accuracy and speed, including parallelizable ray-trace computations. The practical impact is a versatile, machine-precision toolbox for validating probabilistic decision models in uncertainty, enabling precise model comparison, dimension reduction, and diagnostic tests across complex, multi-cue visual tasks.

Abstract

Univariate and multivariate normal probability distributions are widely used when modeling decisions under uncertainty. Computing the performance of such models requires integrating these distributions over specific domains, which can vary widely across models. Besides some special cases, there exist no general analytical expressions, standard numerical methods or software for these integrals. Here we present mathematical results and open-source software that provide (i) the probability in any domain of a normal in any dimensions with any parameters, (ii) the probability density, cumulative distribution, and inverse cumulative distribution of any function of a normal vector, (iii) the classification errors among any number of normal distributions, the Bayes-optimal discriminability index and relation to the operating characteristic, (iv) ways to scale the discriminability of two distributions, (v) dimension reduction and visualizations for such problems, and (vi) tests for how reliably these methods may be used on given data. We demonstrate these tools with vision research applications of detecting occluding objects in natural scenes, and detecting camouflage.

Figures (9)

• Figure 1: Method schematic. a. Standard normal error ellipse is blue. Arrow indicates a ray from it at angle $\bm{n}$ in an angular slice $d\bm{n}$, crossing the gray integration domain $\tilde{f}(\bm{z})>0$ at $\bm{z}_1$ and $\bm{z}_2$. b. 1d slice of this picture along the ray. The standard normal density along a ray is blue. $\tilde{f}_{\bm{n}}(z)$ is the slice of the domain function $\tilde{f}(\bm{z})$ along the ray, crossing 0 at $z_1$ and $z_2$.
• Figure 2: Toolbox outputs for some integration and classification problems. a. Top: the probability of a 3d normal (blue shows 1 sd error ellipsoid) in an implicit toroidal domain $f_t(\bm{x})>0$. Black dots are boundary points within 3 sd traced by the ray method, across Matlab's adaptive integration grid over angles. Inset: pdf of $f_t(\bm{x})$ and its integrated part (blue overlay). Bottom: integrating a 2d normal (blue error ellipse) in a domain built by the union of two circles. b. Estimates of the 4d standard normal probability in the 4d polyhedral domain $f_p(\bm{x})= \sum_{i=1}^4 \lvert x_i \rvert<1$ using the ray-trace method with Monte Carlo ray-sampling, across 5 runs, converging with growing sample size of rays. Inset: pdf of $f_p(\bm{x})$ and its integrated part. c. Left: heat map of joint pdf of two functions of a 2d normal, to be integrated over the implicit domain $f_1-f_2>1$ (overlay). Right: corresponding integral of the normal over the domain $h(\bm{x})=x_1 \sin x_2 - x_2 \cos x_1 >1$ (blue regions), 'traced' up to 3 sd (black dots). Inset: pdf of $h(\bm{x})$ and its integrated part. d. Classifying two 2d normals using the optimal boundary $l$, which yields the Bayes-optimal discriminability $d'_b$. Color-bar shows the proportions by which the two dimensions contribute to the overall discriminability. $d'_e$ and $d'_a$ are approximate discriminability indices. A custom suboptimal boundary such as the green line can also be used for classification. e. Classification based on samples (dots) from non-normal distributions. Filled ellipses are error ellipses of fitted normals. $\gamma$ is an optimized boundary between the samples. The three error rates are: of the normals with $l$, of the samples with $l$, and of the samples with $\gamma$. f. Classifying several 2d normals with arbitrary means and covariances. g. Top: 1d projection of a 4d normal integral over a quadratic domain $q(\bm{x})>0$. Bottom: Projection of the classification of two 4d normals based on samples, with unequal priors, and unequal outcome values (correctly classifying the blue class is valued 4 times the red, hence the optimal criterion is shifted), onto the axis of the Bayes decision variable $\beta$. Histograms and smooth curves are the projections of the samples and the fitted normals. The sample-optimized boundary $\gamma=0$ cannot be uniquely projected to this $\beta$ axis. h. Classification based on four 4d non-normal samples, with different priors and outcome values, projected on the axis along (1,1,1,1). The boundaries cannot be projected to this axis.
• Figure 3: Binary yes/no and two-interval classification tasks. a. Optimal yes/no decision between two unequal-variance 1d normal distributions. b. The same task transformed to the log likelihood ratio axis (log vertical axis for clarity). c. Optimal two-interval discrimination between the same 1d normal distributions $a$ and $b$ is actually a discrimination between 2d normals $ab$ and $ba$. d. The task transformed to the log likelihood ratio axis.
• Figure 4: Comparing discriminability indices. a. Plots of existing indices $d'_a$ and $d'_e$ as fractions of the Bayes index $d'_b$, with increasing separation between two 1d and two 2d normals, for different ratios $s$ of their sd's. b. Left : two normals with 1 sd error ellipses corresponding to their sd matrices $\mathbf{S}_a$ and $\mathbf{S}_b$, and their average and rms sd matrices. Right: the space has been linearly transformed, so that $a$ is now standard normal, and $b$ is aligned with the coordinate axes. c. Discriminating two highly-separated 1d normals.
• Figure 5: ROC curves. a. Yes/no ROC curves for a single shifting criterion (black), vs. a shifting likelihood-ratio (green), between the two 1d normals of fig. \ref{['fig:3_binary_class']}a (adapted from Wickens wickens2002elementary, fig. 9.3), and a shifting likelihood-ratio between a normal and a $t$ distribution in 4d (purple). The optimal two-interval accuracies of the 1d normals (fig. \ref{['fig:3_binary_class']}c) and the 4d distributions (fig. \ref{['fig:5_ROC']}b) are 0.74 and 0.97, equal to the areas under their likelihood-ratio curves here. The points marked on these curves are the farthest from the diagonal, and correspond to the Bayes discriminability. b. Distributions of the log likelihood ratio of the 4d $t$ vs normal distribution. Sweeping the criterion corresponds to moving along the purple ROC curve of a.