Generalization from Low- to Moderate-Resolution Spectra with Neural Networks for Stellar Parameter Estimation: A Case Study with DESI

TL;DR

It is found that simple pre-trained MLPs can provide competitive cross-survey generalization, while the role of spectral foundation models for cross-survey stellar parameter estimation requires further exploration.

Abstract

Cross-survey generalization is a critical challenge in stellar spectral analysis, particularly in cases such as transferring from low- to moderate-resolution surveys. We investigate this problem using pre-trained models, focusing on simple neural networks such as multilayer perceptrons (MLPs), with a case study transferring from LAMOST low-resolution spectra (LRS) to DESI medium-resolution spectra (MRS). Specifically, we pre-train MLPs on either LRS or their embeddings and fine-tune them for application to DESI stellar spectra. We compare MLPs trained directly on spectra with those trained on embeddings derived from transformer-based models (self-supervised foundation models pre-trained for multiple downstream tasks). We also evaluate different fine-tuning strategies, including residual-head adapters, LoRA, and full fine-tuning. We find that MLPs pre-trained on LAMOST LRS achieve strong performance, even without fine-tuning, and that modest fine-tuning with DESI spectra further improves the results. For iron abundance, embeddings from a transformer-based model yield advantages in the metal-rich ([Fe/H] > -1.0) regime, but underperform in the metal-poor regime compared to MLPs trained directly on LRS. We also show that the optimal fine-tuning strategy depends on the specific stellar parameter under consideration. These results highlight that simple pre-trained MLPs can provide competitive cross-survey generalization, while the role of spectral foundation models for cross-survey stellar parameter estimation requires further exploration.

Figures (11)

• Figure 1: Sketch of the pre-training and fine-tuning workflow. Pre-training uses normalized spectra or foundation-model spectral embeddings to predict labels: [Fe/H] from APOGEE ($>$ –2.0), supplemented at lower metallicities by PASTEL, SAGA, and other VMP/UMP datasets; [$\alpha$/Fe] from APOGEE. The foundation model is trained on large unlabeled datasets. Fine-tuning then adapts the network using fewer labeled spectra, either from spectra or spectral embeddings, with different fine-tuning modules applied (see Section \ref{['sec:finetune_mlp']}).
• Figure 2: Comparison of [Fe/H] and [$\alpha$/Fe] estimates between the DESI SP pipeline and MLP-based models, referenced against APOGEE DR17 labels. From left to right: DESI SP--DESI SP pipeline, MLP-Scratch--MLP trained from scratch on the same number of DESI spectra used for fine-tuning, MLP-LRS--MLP pre-trained on LAMOST LRS (zero-shot application), and MLP-LRS (fine-tuned)--MLP pre-trained on LAMOST LRS and fine-tuned with DESI spectra. The first row shows [Fe/H] results; the second row shows [$\alpha$/Fe]. Legends list the number of test stars (N), coefficient of determination ($\mathrm{R^2}$), and the robustly estimated standard deviation of the residuals ($\sigma$, computed after 3$\sigma$ clipping with sigma_clip in astropy). All sources are restricted to $T_{\rm eff}>4000$ K. The dashed black line indicates the ideal 1:1 relation, and the gray dotted lines mark ±0.2 dex deviations. The fine-tuning set consists of 2,069 stars. A comparison using the clean, calibrated DESI SP subset is provided in Appendix \ref{['sec:compare_clean']}.
• Figure 3: [$\alpha$/Fe]–[Fe/H] diagrams for DESI EDR and APOGEE reference sample. From left to right: (1) DESI SP: DESI SP pipeline, (2) MLP-Scratch: MLP trained from scratch on the same number of DESI spectra used for fine-tuning, (3) MLP-LRS: MLP pre-trained on LAMOST LRS (zero-shot application), (4) MLP-LRS (fine-tuned): MLP pre-trained on LAMOST LRS and fine-tuned with DESI spectra, and (5) APOGEE: DESI DR1–APOGEE DR17 cross-matched sample (7,608 stars). The first row shows raw counts, and the second row shows column-normalized densities, where each [Fe/H] column is divided by its maximum count. The legend reports the Maximum Mean Discrepancy (MMD) between each sample and the APOGEE reference.
• Figure 4: Effect of fine-tuning sample size (0, 100, 200, 500, 1000, and 2069, where 0 denotes zero-shot performance), evaluated on the testing sample. Shown are $R^2$ and the robustly estimated standard deviation of the residuals ($\sigma$) for [Fe/H] and [$\alpha$/Fe], obtained after fine-tuning (residual-head fine-tuning for [Fe/H] and LoRA fine-tuning for [$\alpha$/Fe]). For both metrics, we plot the mean and 1$\sigma$ error bars from five independent runs. The scatter plots for the different sample sizes are shown in Figure \ref{['fig:sample_size_one2one']} and \ref{['fig:sample_size_2d']}.
• Figure 5: Loss landscapes for different fine-tuning strategies, evaluated on the metal-poor regime of [Fe/H] (79 stars). The first row shows 3D surfaces, and the second row shows 2D contours of the logarithmic MSE loss, plotted as a function of perturbations around the pre-trained model parameters along two random directions. The minimum losses are indicated in the colorbar of the 2D contours.