QUEST (Quasar Unsupervised Encoder and Synthesis Tool): A machine learning framework to generate quasar spectra

TL;DR

QUEST presents an Information Maximising Variational Auto-Encoder that learns from SDSS DR16Q to generate realistic quasar spectra across a broad rest-frame range. The approach yields three complementary training datasets (GP, FOB, FOR) and demonstrates synthetic photometry, BAL imputation, and emission-line-based black hole mass estimates, while mapping latent space to physical properties via visual, UMAP, and mutual-information analyses. The results show accurate reproduction of input spectra, credible photometry in SDSS and high-z regimes, and plausible imputation of absorption features, with Lyman-α forest reconstruction providing competitive continuum estimates. This framework enables scalable, data-driven generation and conditioning of quasar spectra for upcoming surveys, with future plans to broaden training data, extend wavelength coverage, and add conditional generation for targeted quasar populations.

Abstract

Quasars at the redshift frontier (z > 7.0) are fundamental probes of black hole (BH) growth and evolution but notoriously difficult to identify. At these redshifts, machine learning-based selection methods have proven to be efficient, but require appropriate training sets to express their full potential. Here, we present QUEST, a Variational Auto-Encoder capable of generating realistic quasar spectra that can be post-processed for generating synthetic photometry and for spectral imputation. We start from the SDSS DR16Q catalogue, pre-process the spectra, and vet the sample to obtain a clean data set. After training the model, we investigate the properties of its latent space to understand whether it has learnt relevant physics. We provide a pipeline to generate photometry from the sampled spectra, compare it with actual quasar photometry, and showcase the capabilities of the model in reconstructing and extending quasar spectra. The trained network faithfully reproduces the input spectrum, both in terms of sample median and variance. By examining the latent space, we find correlations with continuum and bolometric luminosity, BH mass, redshift, continuum slope, and emission line properties. When used to generate photometry, we find results in excellent agreement with the control sample. The model provides satisfactory results in reconstructing emission lines: estimates of the BH mass from the reconstructed spectra are in good agreement with those from the original spectra. Furthermore, when spectra with broad absorption line features are reconstructed, the model successfully interpolates over the absorption systems. Compared with previous work, we find excellent agreement between the spectra sampled from our model and the output of their results. However, QUEST does not require any ad-hoc tuning, and is capable of reproducing the full variety of spectra available in the training set.

Figures (22)

• Figure 1: Top panel: Example of SDSS spectra at different redshifts included in the GP dataset. Bottom panel: Example of spectrum from the GNIRS-DQS survey, highlighting the extended coverage at redder wavelengths. Spectral gaps are are due to telluric absorptions.
• Figure 2: Examples of rejected spectra, and the cause of rejection. The black line shows the original SDSS spectrum, the red one the nominal uncertainty and the light grey one the zero-flux level.
• Figure 3: Density plot showing the redshift-absolute i-band magnitude distribution for the spectra that meet the selection criteria. The colour map shows the number of spectra in each contour line.
• Figure 4: Median spectrum and logarithmic number of spectra contributing to the median in each pixel. Top panel: median spectrum of the quasars included in the training set (black, thick, solid line) compared to the vanden_berk_composite_2001 (red, thin, solid line). Shaded regions represent the 16$^{\rm th}$--84$^{\rm th}$ percentiles (dark grey) and the 1$^{\rm st}$--99$^{\rm th}$ percentile (light grey). The vertical, blue band represents the region in which we compute the normalisation factor for each spectrum. Bottom panel: logarithmic number of spectra contributing to each pixel in the median spectrum. By construction, all spectra contribute to the region between 2300--2600 Å.
• Figure 5: Schematic representation of the model architecture, input and output. The network receives as input the concatenation of an SDSS spectrum, normalised by the median spectrum, and the coverage mask (top left panel). It then encodes the input to produce a latent space representation $\mathcal{Z}$, that is decoded to produce a new spectrum (top right panel). The new spectrum (shown in red) covers a wider wavelength range compared to the corresponding input (shown in black) and is generally less noisy. The encoder and decoder are built as the reverse of each other by combining several hidden layers, denoted in the schematic by HL followed by the corresponding output dimension. Each hidden layer is a combination of a linear layer, followed by a BatchNorm1D layer and the activation function (bottom left).