Reducing false positives in strong lens detection through effective augmentation and ensemble learning

TL;DR

The paper tackles false positives in automated strong gravitational lens detection by examining how training-data quality and diversity shape CNN performance. It systematically compares Vanilla, Applied1, and Applied2 data strategies and employs ensembles of DenseNet and EfficientNet architectures to reduce false positives while preserving completeness. A Combined ensemble achieves a false-positive rate of $10^{-4}$ with a modest ~2.3 percent drop in true positives, illustrating a favorable FP-TP trade-off for large surveys like KiDS and Euclid. The work provides actionable guidance on data-design and ensembling to enable scalable, reliable lens candidate catalogs for current and upcoming cosmological surveys.

Abstract

This research studies the impact of high-quality training datasets on the performance of Convolutional Neural Networks (CNNs) in detecting strong gravitational lenses. We stress the importance of data diversity and representativeness, demonstrating how variations in sample populations influence CNN performance. In addition to the quality of training data, our results highlight the effectiveness of various techniques, such as data augmentation and ensemble learning, in reducing false positives while maintaining model completeness at an acceptable level. This enhances the robustness of gravitational lens detection models and advancing capabilities in this field. Our experiments, employing variations of DenseNet and EfficientNet, achieved a best false positive rate (FP rate) of $10^{-4}$, while successfully identifying over 88 per cent of genuine gravitational lenses in the test dataset. This represents an 11-fold reduction in the FP rate compared to the original training dataset. Notably, this substantial enhancement in the FP rate is accompanied by only a 2.3 per cent decrease in the number of true positive samples. Validated on the KiDS dataset, our findings offer insights applicable to ongoing missions, like Euclid.

Figures (14)

• Figure 1: These images illustrate the dataset's diversity, presenting lensed phenomena and non-lensed galaxies. The top row showcases various lensed samples, offering insights into their morphology and configurations. The middle panel displays a selection of LRGs, serving as both foreground and non-lensed instances, while the bottom panel includes spiral galaxies. More details about the properties of these galaxies are available in Section \ref{['training_data']} and Table \ref{['tab:sim_params']}. Each image has a size of $101 \times 101$ pixels, which corresponds to an area of $20 \times 20$ arcsec.
• Figure 2: The distribution of the lensing galaxy model parameters used for the training dataset; lensing galaxy effective radius (top) and the lensing galaxy complexity (bottom).
• Figure 3: The distribution of the lens model parameters used for the training dataset; these are (left panel) the lens axis ratio (b/a), (middle panel) the lens external shear ($\gamma_{\rm ext}$ ), and (right panel) the lens Einstein radius ($\theta_{\rm E}$ ). Notably, the Einstein radii follow a logarithmic distribution, while the other parameters adhere to a flat distribution. The position angles of the ellipsoidal mass distribution and the external shear were set randomly between $\pm 90$ deg.
• Figure 4: An example of a potential issue arising when the lensed emission is faint with respect to the brightness of the foreground lensing galaxy, and has an Einstein radius that is significantly lower than the effective radius of the foreground lensing galaxy. Two scenarios employing the same LRG as the foreground galaxy are shown, but with different lens configurations. As depicted, when lensed emission with a smaller Einstein radius is introduced to the selected LRG, the ring configuration of the lensed emission is entirely obscured by the LRG emission. The inclusion of lensed samples, such as the example on the left (labeled as 1 or lensed), may confuse the CNN model, as this sample resembles a non-lensed sample (labeled as 0) in our training dataset.
• Figure 5: The ROC plot demonstrates how different machine learning models, trained with the Vanilla setting, perform in distinguishing between positive (lensed) and negative (non-lensed) samples across various thresholds balancing true positive (TP) and false positive (FP) rates. For presentation purposes, the range of the ROC plot has changed from [0,1] to the current display. By comparing these curves, we can identify the most efficient model for detecting lensed samples, while minimizing FPs. The plotted results show the improvement in FP rate as we use an ensemble technique, by averaging the predicted lens probability of individual models. The best performing ensemble belongs to averaging the output of EfficientNet-B3, EfficientNet-B4 and DenseNet-121 with the FP rate of $1.1 \times 10^{-3}$ and a TP rate of 0.906.