Simulation-Based Pretraining and Domain Adaptation for Astronomical Time Series with Minimal Labeled Data

TL;DR

The paper tackles the data scarcity problem in astronomical time-series analysis by introducing a simulation-based pretraining framework that spans multiple surveys (ZTF and LSST) to learn domain-agnostic representations. It evaluates three model variants (base classifier, adversarial, and supervised contrastive) on real ZTF data, simulated redshift data, and Kepler cross-domain data, showing substantial gains in classification, redshift estimation, and anomaly detection with limited labeled data and notable zero-shot transfer to LSST and cross-domain generalization to Kepler. Key findings include that limited-label fine-tuning with pretrained models can outperform fully supervised baselines and that domain-agnostic objectives enable effective zero-shot transfer, even across substantially different observational setups. The approach has immediate practical impact for LSST readiness, enabling science from day one without extensive domain-specific labeling and offering a principled framework for other scientific domains where simulations encode domain expertise but labeled data remain scarce.

Abstract

Astronomical time-series analysis faces a critical limitation: the scarcity of labeled observational data. We present a pre-training approach that leverages simulations, significantly reducing the need for labeled examples from real observations. Our models, trained on simulated data from multiple astronomical surveys (ZTF and LSST), learn generalizable representations that transfer effectively to downstream tasks. Using classifier-based architectures enhanced with contrastive and adversarial objectives, we create domain-agnostic models that demonstrate substantial performance improvements over baseline methods in classification, redshift estimation, and anomaly detection when fine-tuned with minimal real data. Remarkably, our models exhibit effective zero-shot transfer capabilities, achieving comparable performance on future telescope (LSST) simulations when trained solely on existing telescope (ZTF) data. Furthermore, they generalize to very different astronomical phenomena (namely variable stars from NASA's \textit{Kepler} telescope) despite being trained on transient events, demonstrating cross-domain capabilities. Our approach provides a practical solution for building general models when labeled data is scarce, but domain knowledge can be encoded in simulations.

Figures (4)

• Figure 1: Overview of our simulation-based pre-training methodology showing three model variants (Classifier, Adversarial, Contrastive) and evaluation on both in-domain and cross-domain tasks. We first pre-train various classifiers and domain-agnostic models using simulated data. We then evaluate these models on various downstream tasks, including zero-shot estimation for new telescopes.
• Figure 2: UMAP visualization of learned representations across all classes. The standard classifier maintains distinct ZTF (brown) and LSST (blue) clusters, while domain-agnostic training methods appear to unify surveys within each astronomical class slightly better.
• Figure 3: Class-level alignment for Type II supernovae only. Domain-agnostic methods achieve better survey overlap in the latent representations while preserving astronomical class structure, indicating at possible cross-survey transfer capabilities.
• Figure 4: Redshift distributions across datasets. (a) Simulated ZTF transients span $z = 0.0-0.4$ while LSST simulations extend to $z \sim 3.0$, reflecting LSST's deeper photometric sensitivity ($r \sim 25$ mag) compared to ZTF ($r \sim 21$ mag). (b) Real ZTF observations show a similar distribution to ZTF simulations, validating the simulation fidelity. For zero-shot evaluation, we restrict LSST objects to ZTF's redshift range ($z < 0.4$) to test transfer learning within the training distribution.