Scaling laws for amplitude surrogates

TL;DR

This work investigates how neural surrogates for scattering amplitudes in particle physics scale with training data, network size, and compute. By analyzing both MLP-I and LLoCa-Transformer architectures across multiple processes, it shows that scaling exponents are governed by the intrinsic dimension $d$ of the task, with $d = 3n_f - 4$ for amplitudes, and that exponents satisfy approximate bounds $alpha_N, alpha_D, alpha_C \approx 4/d$. The study demonstrates power-law scaling across a broad set of processes and losses, and provides a practical framework to estimate the resources needed to achieve a target precision, including a method to anchor scaling curves using low-cost surrogates. It also reveals that the learned representation dimension tracks the degrees of freedom of the underlying amplitudes, suggesting that neural surrogates can quantify intrinsic data-space structure and inform efficient event-generation pipelines.

Abstract

Scaling laws describing the dependence of neural network performance on the amount of training data, the spent compute, and the network size have emerged across a huge variety of machine learning task and datasets. In this work, we systematically investigate these scaling laws in the context of amplitude surrogates for particle physics. We show that the scaling coefficients are connected to the number of external particles of the process. Our results demonstrate that scaling laws are a useful tool to achieve desired precision targets.

Figures (20)

• Figure 1: Left: MSE test error for the $q\bar{q}\to t\bar{t} H$ amplitude using the MLP-I model as a function of the the initial learning rate for different network sizes. Right: Same as left, but the test error is shown as a function of the compute in FLOPs for different training dataset sizes. The fitted power-law functions are depicted as shaded lines.
• Figure 2: Left: MSE test error for the $q\bar{q}\to t\bar{t} H$ amplitude using the MLP-I model as a function of the training dataset size for different NN sizes. The fitted power-law functions are depicted as shaded lines. Right: Same as left, but the test error is shown as a function of the NN size for different training dataset sizes.
• Figure 3: Left: Intrinsic dimension extracted by the $q\bar{q}\to t\bar{t}H$ surrogate as a function of the training dataset size for different NN sizes. The color bounds indicate the associated uncertainty. The intrinsic dimension is calculated based on a subset of the final-layer activations with the uncertainties corresponding to the standard deviation obtained by sampling different subsets. Right: Same as left, but the intrinsic dimension is shown as a function of the NN size for different training dataset sizes.
• Figure 4: Left: MSE test error for the $q\bar{q}\to t\bar{t} H$ amplitude using the MLP-I model trained with a heteroscedastic loss as a function of the compute in FLOPs for different training dataset sizes. The fitted power-law functions are depicted as shaded lines. Right: Same as left, but the test error is shown a function of the training dataset size for different training dataset sizes for different NN sizes.
• Figure 5: Left: Distribution of learned uncertainty values for the $q\bar{q}\to t\bar{t}H$ surrogate for different amounts of spent compute. The dataset size is fixed to $D_\text{train}=3.16\times 10^5$. Right: Same as left, but the distributions are shown for different sizes of the training dataset. The compute is fixed to $C=2.12\times 10^{15}$ FLOPs.