A case study of sending graph neural networks back to the test bench for applications in high-energy particle physics

TL;DR

This study benchmarks graph neural networks against fully-connected neural networks for classifying the underlying ttbb-related processes (ttbb, ttH(bb), ttZ(bb)) in LHC-like events, under a carefully controlled setup that fixes hyperparameters and information exposure. The authors show that GNNs do not inherently outperform DNNs without relational information; incorporating physics-m informed edge weights and multiple GraphConv layers allows GNNs to recover or surpass DNN performance, particularly when the models have comparable numbers of trainable parameters. The results highlight that the gain from GNNs arises from access to hierarchical neighbor information encoded by graph convolutions, rather than mere graph structure. The findings suggest that GNNs can offer advantages in tasks involving hierarchically structured, multi-object final states, with implications for jet physics and event classification at the LHC. Overall, the work provides a principled framework to assess GNNs in high-energy physics and clarifies when relational inductive biases are beneficial.

Abstract

In high-energy particle collisions, the primary collision products usually decay further resulting in tree-like, hierarchical structures with a priori unknown multiplicity. At the stable-particle level all decay products of a collision form permutation invariant sets of final state objects. The analogy to mathematical graphs gives rise to the idea that graph neural networks (GNNs), which naturally resemble these properties, should be best-suited to address many tasks related to high-energy particle physics. In this paper we describe a benchmark test of a typical GNN against neural networks of the well-established deep fully-connected feed-forward architecture. We aim at performing this comparison maximally unbiased in terms of nodes, hidden layers, or trainable parameters of the neural networks under study. As physics case we use the classification of the final state X produced in association with top quark-antiquark pairs in proton-proton collisions at the Large Hadron Collider at CERN, where X stands for a bottom quark-antiquark pair produced either non-resonantly or through the decay of an intermediately produced Z or Higgs boson.

Figures (8)

• Figure 1: Exemplary Feynman diagrams for the processes of interest to this study: (left) $\mathrm{ttbb}$, (middle) $\mathrm{ttH(bb)}$, and (right) $\mathrm{ttZ(bb)}$.
• Figure 2: Translation of an (upper part) selected event (without jets in the NA jet-class in this case) into a (middle part) graph $\mathcal{G}$ and finally into the (lower part) GNN model. For the indication of the partonic final state we do not distinguish particles from anti-particles. The individual object-classes are indicated by different colors. The nodes of $\mathcal{G}$ are labelled by $i=1\ldots 8$ and colored the same way as the object-classes. The boxes next to the nodes indicate the embedding space of the GNN model. The GNN output is indicated by $\hat{y}$.
• Figure 3: Mean ROC-AUC $\mu_{\mathrm{AUC}}$ as obtained for 18 different configurations of GNN and 24 corresponding configurations of DNN models with $k\xspace=1$. The labels in brackets on the vertical axis indicate the use of relational information, as discussed in Section \ref{['sec:feature_space']}, the numbers in parentheses correspond to the choices of $n$. The circles refer to GNN and the upward (downward) pointing triangles to DNN models with a default (reduced) set of input features $\mathbf{x}^{\mathrm{DNN}}$ ($\mathbf{x}^{\mathrm{DNN}}\xspace_{\mathrm{red}}$), as discussed in Section \ref{['sec:feature_space']}. For better readability, markers of the same configuration are shifted vertically along the y-axis. The bars are obtained from the sample variance of an ensemble, as described in Section \ref{['subsec:training_setup']}. Those NN architectures which belong to the same choices of varying parameters are spatially grouped and shown in the same color. Open markers indicate that significant outliers of the corresponding distribution of ROC-AUC values have been removed from the calculation of $\mu_{\mathrm{AUC}}$ and its variance, as described in the text.
• Figure 4: Mean ROC-AUC $\mu_{\mathrm{AUC}}$ as obtained for 54 different GNN and 72 corresponding DNN models with $k\xspace=2$. The labels in brackets on the vertical axis indicate the use of relational information, as discussed in Section \ref{['sec:feature_space']}, the numbers in parentheses correspond to $n$. The circles refer to GNN and the upward (downward) pointing triangles to DNN models with a default (reduced) set of input features $\mathbf{x}^{\mathrm{DNN}}$ ($\mathbf{x}^{\mathrm{DNN}}\xspace_{\mathrm{red}}$), as discussed in Section \ref{['sec:feature_space']}. For better readability, markers of the same configuration are shifted vertically along the y-axis. The bars are obtained from the sample variance of an ensemble, as described in Section \ref{['subsec:training_setup']}. NN architectures which belong to the same choices of varying parameters are spatially grouped and shown in the same color. Open markers indicate that significant outliers of the corresponding distribution of ROC-AUC values have been removed from the calculation of $\mu_{\mathrm{AUC}}$ and its variance, as described in the text.
• Figure 5: Summary of the achieved values of $\mu_{\mathrm{AUC}}$ for the GNN models with (upper half) one and (lower half) two GraphConv operations, with different use of relational information. For this summary, the associations of $n$ with the highest values of $\mu_{\mathrm{AUC}}$ in each group of GNN models have been used. The value of $\mu_{\mathrm{AUC}}$ is displayed on the $x$-axis. Improvements relative to the least separating GNN with no relational information at all (zero) is given in numbers to the right of the bars. The use of relational information, as defined in Table \ref{['tab:parameter_choices']}, is indicated in brackets, on the $y$-axis.