Exploration of Parameter Spaces Assisted by Machine Learning

TL;DR

This work tackles the challenge of exploring high-dimensional parameter spaces where expensive calculations hinder direct sampling, by introducing two iterative ML strategies: regression-based sampling that predicts observables $Y(K)$ and classifier-based sampling that forecasts region membership with predictions in $[0,1]$. The methods are benchmarked on a three-dimensional toy model with shell-like regions defined by $\mathcal{L}_{3d}$ and on a seven-parameter type II 2HDM scan using HiggsBounds/HiggsSignals constraints. Results show that the regressor and classifier achieve comparable or superior coverage to MCMC and MultiNest, with the classifier enhanced by SMOTE boosting accelerating initial convergence. The approach yields efficient, uniform sampling of complex regions and produces physically informative parameter ranges and mass spectra, with code publicly available.

Abstract

We demonstrate two sampling procedures assisted by machine learning models via regression and classification. The main objective is the use of a neural network to suggest points likely inside regions of interest, reducing the number of evaluations of time consuming calculations. We compare results from this approach with results from other sampling methods, namely Markov chain Monte Carlo and MultiNest, obtaining results that range from comparably similar to arguably better. In particular, we augment our classifier method with a boosting technique that rapidly increases the efficiency within a few iterations. We show results from our methods applied to a toy model and the type II 2HDM, using 3 and 7 free parameters, respectively. The code used for this paper and instructions are publicly available on the web.

Figures (9)

• Figure 1: Top: Charts for the ML iterative process used for the regressor (left) and the classifier (right). The main predict-train loop is indicated with black arrows, green arrows indicate places where a random number set is required and the orange arrow marks where we collect the output points. The points accumulated up to step $n_k$ are added for training during step $n_{k+1}$. Bottom: The neural network used for the regressor and classifier will take model parameters as input. In the case of the regressor, returns observables, likelihood, or any other results that were used for training. For the classifier, it returns the class probability. The neural network we use in the examples of the text is a multilayer perceptron (MLP).
• Figure 2: Coverage for 13 different regions of the toy model with 20 000 points, using DNNR (top left), DNNC (top right), MultiNest (middle left) and MCMC (middle right). The color indicates deviation from average distribution of points per region (with average normalized to 1). Best result (less deviation) out of 10 runs is displayed. In the bottom panel, in light color, we show standard deviations for the 13 regions averaged over 10 runs. Again, the average number of points per region has been normalized to 1. The markers show the distribution of the best result out of 10, corresponding to the points in the top 4 panels.
• Figure 3: Result of accumulating 20 000 points with $\mathcal{L}_{N\text{d}} > 0.01$ for the toy model described in Sec. \ref{['sec:toyNdimensions']} using the regressor. Only the first three dimension are shown that are shared with the 3-dimensional toy model of Eq. \ref{['eq:3dmodel']}. The model explored in the left panel has 20 dimensions while for the right panel it has 40 dimensions. A slight transparency has been applied to show parts with deficient coverage.
• Figure 4: Effect of the SMOTE technique on the toy model of Eq. \ref{['eq:3dmodel']}. On the upper panels, an initial set of 100 points (left, dark blue) in the target region is increased to 4743 (right, dark blue and green). In the bottom panels, we apply SMOTE to the $x_1 = 0$ slice of Eq. \ref{['eq:3dmodel']} to increase 100 points in the target region to 4517, where colors have identical meaning to the upper panels. In both upper and bottom panels the SMOTE technique boosts the minority class using 3 nearest neighbors. The proportion of points suggested by SMOTE that are actually in the target region is around 50% for the upper panels and around 90% for the bottom panels. The light red points indicate the target region in all the panels.
• Figure 5: Number of accumulated points in terms of iterations when using (blue) and not using (orange) the boosting method, for the 3-dimensional toy model. The sizes of the sets $K_0$, $L$ and $K$ as well as details for the selection of points are described in the text.