Conditional Neural Bayes Ratio Estimation for Experimental Design Optimisation

Abstract

For frontier experiments operating at the edge of detectability, instrument design directly determines the probability of discovery. We introduce Conditional Neural Bayes Ratio Estimation (cNBRE), which extends neural Bayes ratio estimation by conditioning on design parameters, enabling a single trained network to estimate Bayes factors across a continuous design space. Applied to 21-cm radio cosmology with simulations representative of the REACH experiment, the amortised nature of cNBRE enables systematic design space exploration that would be intractable with traditional point-wise methods, while recovering established physical relationships. The analysis demonstrates a ~20 percentage point variation in detection probability with antenna orientation for a single night of observation, a design decision that would be trivial to implement if determined prior to antenna construction. This framework enables efficient, globally-informed experimental design optimisation for a wide range of scientific applications.

Figures (5)

• Figure 1: Detection probability as a function of observation time $\tau$ for three Bayes factor thresholds corresponding to $1\sigma$ ($K\approx 3$), $3\sigma$ ($K\approx 150$), and $5\sigma$ ($K\approx 1.7\times 10^6$) significance.
• Figure 2: Detection probability as a function of observation time ($\tau$) and signal amplitude ($A$). Detection probabilities are computed on test sets with signals injected at fixed amplitudes; the network marginalizes over the full signal prior during training. The panels show $5\sigma$, $3\sigma$, and $1\sigma$ thresholds, confirming that detection probability improves with longer integration and larger amplitudes.
• Figure 3: Multi-panel visualisation of detection probability as a function of observation time ($\tau$) and signal central frequency ($f_0$). Detection probabilities are computed on test sets with signals injected at fixed $f_0$ values; the cNBRE network itself marginalizes over the full signal prior during training. The three heatmaps show $5\sigma$, $3\sigma$, and $1\sigma$ thresholds (left to right). Marginal plots highlight reduced sensitivity at the band edges.
• Figure 4: Polar heatmap showing the variation in detection probability (radial axis) and achievable significance (colour scale) as a function of antenna orientation angle (angular axis) for several observations over a single night. The four-fold symmetry reflects the interaction of the rectangular dipole beam with galactic foregrounds.
• Figure 5: Comparison of log-Bayes factors estimated by cNBRE (y-axis) versus nested sampling (x-axis) for 1,600 simulated data realisations (811 $M_1$, 789 $M_0$). Circles denote $M_1$ samples (signal present); crosses denote $M_0$ samples (null model). The dashed line indicates perfect agreement. Both methods show broad agreement in the decision-relevant regime, with $M_0$ samples clustering at negative $\log K$ and $M_1$ samples at positive $\log K$ for both estimators. At very high Bayes factors the cNBRE estimates saturate below the nested sampling values, a consequence of classifier output saturation that is expected and practically irrelevant for detection decisions.