CLAP. I. Resolving miscalibration for deep learning-based galaxy photometric redshift estimation

TL;DR

A novel method is developed called C ontrastive L earning and A daptive KNN for P hotometric Redshift (CLAP) that resolves the issue of miscalibration and demonstrates the robustness of CLAP for obtaining photometric redshift probability densities required by astrophysical and cosmological applications.

Abstract

Obtaining well-calibrated photometric redshift probability densities for galaxies without a spectroscopic measurement remains a challenge. Deep learning discriminative models, typically fed with multi-band galaxy images, can produce outputs that mimic probability densities and achieve state-of-the-art accuracy. However, such models may be affected by miscalibration that would result in discrepancies between the model outputs and the actual distributions of true redshifts. Our work develops a novel method called the Contrastive Learning and Adaptive KNN for Photometric Redshift (CLAP) that resolves this issue. It leverages supervised contrastive learning (SCL) and k-nearest neighbours (KNN) to construct and calibrate raw probability density estimates, and implements a refitting procedure to resume end-to-end discriminative models ready to produce final estimates for large-scale imaging data. The harmonic mean is adopted to combine an ensemble of estimates from multiple realisations for improving accuracy. Our experiments demonstrate that CLAP takes advantage of both deep learning and KNN, outperforming benchmark methods on the calibration of probability density estimates and retaining high accuracy and computational efficiency. With reference to CLAP, we point out that miscalibration is particularly sensitive to the method-induced excessive correlations among data instances in addition to the unaccounted-for epistemic uncertainties. Reducing the uncertainties may not guarantee the removal of miscalibration due to the presence of such excessive correlations, yet this is a problem for conventional deep learning methods rather than CLAP. These discussions underscore the robustness of CLAP for obtaining photometric redshift probability densities required by astrophysical and cosmological applications. This is the first paper in our series on CLAP.

Figures (14)

• Figure 1: Distributions of spectroscopic redshift and $r$-band magnitude for the SDSS, CFHTLS, and KiDS datasets used in our work.
• Figure 2: Graphic illustration of our method CLAP for photometric redshift probability density estimation. The development of a CLAP model consists of several procedures, including supervised contrastive learning (SCL), adaptive KNN, reconstruction, and refitting. The SCL framework is based on neural networks. It uses an encoder network to project multi-band galaxy images and additional input data (i.e. galactic reddening $E(B-V)$, magnitudes) to low-dimensional latent vectors, which form a latent space that encodes redshift information and has a distance metric defined. The spectroscopic redshift labels are leveraged to supervise the redshift outputs predicted by an estimator network for extracting redshift information, but the trained estimator and its outputs are no longer used once the latent space is established (indicated by the shaded region). These outputs are uncalibrated, and should not be regarded as the final estimates produced by CLAP. The adaptive KNN and the KNN-enabled recalibration are implemented locally on the latent space via diagnostics with the probability integral transform (PIT), constructing raw probability density estimates using the known redshifts of the PIT-selected nearest neighbours. The raw estimates are then used as labels to retrain the estimator from scratch in a refitting procedure with the trained encoder fixed, resuming an end-to-end model ready to process imaging data and produce the desired estimates. Lastly, we combine the estimates from an ensemble of CLAP models developed following these procedures (not shown in the figure).
• Figure 3: Supervised contrastive learning (SCL) framework. It contains an encoder, an estimator, and a decoder. The encoder, the same as that shown in Fig. \ref{['fig:clap']}, takes multi-band galaxy images and additional data as inputs, and produces two vectors $v_A$ and $v_B$. The vector $v_A$ is used to encode redshift information and is referred to as the 'latent vector' throughout this work. It is inputted to the estimator that produces a redshift output supervised by the spectroscopic redshift label for extracting redshift information. The concatenation of $v_A$ and $v_B$ is inputted to the decoder to reconstruct images that resemble the input images. With the reconstructed images as inputs, this process is conducted again using the three networks with shared weights, producing the vector $v'_A$. Furthermore, the images reshaped with random flipping and rotation by 90 deg steps are exploited as inputs, producing the vector $v^*_A$. For contrastive learning, the contrast between $v'_A$ and $v_A$ and the contrast between $v^*_A$ and $v_A$ for the same galaxy are minimised (i.e. positive pairs), which are characterised by the Euclidean distance. The contrast between the latent vectors for any two different galaxies is maximised (i.e. a negative pair).
• Figure 4: Distribution offset analysis for the SDSS, CFHTLS, and KiDS target samples. The results obtained by CLAP are compared with the prediction from our supervised contrastive learning (SCL) framework, and those produced by a few benchmark image-based methods including inception networks from Pasquet2019 and Treyer et al. (in prep.), network-based bias correlation from Lin2022, a capsule network from Dey2022, a VGG network, and a hybrid network from Li2022GaZNets, and the photometry-only methods including KNN & regression from Beck2016, Le Phare Arnouts1999Ilbert2006, as well as MLPQNA Brescia2014Cavuoti2015Cavuoti2017, ANNz2 Sadeh2016Bilicki2018, and BPZ BPZ2000 retrieved from the public KiDS catalogues. For the KiDS dataset, the results obtained by CLAP but without the NIR data are also shown ('No NIR'). For both CLAP and the SCL prediction, the ensemble of ten probability density estimates are combined using the harmonic mean. Upper panels:$z - z_{photo}$ distributions. The solid curves show the stacked recentred probability densities $p_{\sum}(z - z_{photo})$ for the methods that have probability density estimation. The dashed curves show the $z_{spec} - z_{photo}$ histograms for all the methods. The error bands are estimated using bootstrap. The vertical dashed lines show the median of $z_{spec} - z_{photo}$ for each method, indicating the centre of each $z_{spec} - z_{photo}$ distribution regardless of the random errors due to limited statistics at high redshift. Lower panels: Cumulative $z - z_{photo}$ distribution offsets. The solid curves show the cumulative offset of the stacked recentred probability density $p_{\sum}(z - z_{photo})$ relative to the corresponding $z_{spec} - z_{photo}$ histogram for each method that has a probability density estimation (i.e. $\Delta F_{p_{\sum}-h}$). The dashed curves show the cumulative offset of the $z_{spec} - z_{photo}$ histogram for each of all the methods (including No-NIR CLAP for the KiDS dataset) relative to that obtained by default CLAP (i.e. $\Delta F_{h-h}$). The error bands are estimated using bootstrap. The vertical dashed line in each panel indicates the zero point where $z$ and $z_{photo}$ coincide.
• Figure 5: Distributions of summary statistics of photometric redshift probability density estimates illustrated for the SDSS, CFHTLS, and KiDS target samples. Comparison is made among the results from CLAP and the methods for which the probability density estimates are available, as in Fig. \ref{['fig:main_pdfsc']}. The definitions of the shown summary statistics can be found in Table \ref{['tab:metrics']}. For both CLAP and the SCL prediction, the ensemble of ten probability density estimates are combined using the harmonic mean. For the PIT, the $y$-axis in each panel is truncated at 0.5 for clearer illustration; the horizontal dashed lines indicate normalised uniform distributions. The PIT distributions produced by MLPQNA and BPZ are dramatically non-flat and are not shown in the figure. For the kurtosis, the vertical dashed lines indicate the value for Gaussian-like distributions (i.e. 3).