Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Universal Spectral Tokenization via Self-Supervised Panchromatic Representation Learning

Jeff Shen, Francois Lanusse, Liam Holden Parker, Ollie Liu, Tom Hehir, Leopoldo Sarra, Lucas Meyer, Micah Bowles, Sebastian Wagner-Carena, Sebastian Wagner-Carena, Helen Qu, Siavash Golkar, Alberto Bietti, Hatim Bourfoune, Nathan Cassereau, Pierre Cornette, Keiya Hirashima, Geraud Krawezik, Ruben Ohana, Nicholas Lourie, Michael McCabe, Rudy Morel, Payel Mukhopadhyay, Mariel Pettee, Bruno Régaldo-Saint Blancard, Kyunghyun Cho, Miles Cranmer, Shirley Ho

TL;DR

The paper addresses the challenge of unifying heterogeneous, multi-resolution spectral data across astronomical surveys into a single, reusable representation. It introduces a Vision Transformer–based universal spectrum tokenizer that ingests native wavelength grids and learns wavelength-aware embeddings via self-supervised pretraining on SDSS, DESI, GALAH, and APOGEE. Key contributions include a novel architecture that operates without homogenization, a reconstruction objective, and demonstrations that lightweight adapters achieve competitive performance on tasks such as object classification and stellar parameter estimation. The approach enables cross-survey transfer and foundation-model–like scalability for astronomy, with potential extension to other scientific domains featuring irregular sequential data.

Abstract

Sequential scientific data span many resolutions and domains, and unifying them into a common representation is a key step toward developing foundation models for the sciences. Astronomical spectra exemplify this challenge: massive surveys have collected millions of spectra across a wide range of wavelengths and resolutions, yet analyses remain fragmented across spectral domains (e.g., optical vs. infrared) and object types (e.g., stars vs. galaxies), limiting the ability to pool information across datasets. We present a deep learning model that jointly learns from heterogeneous spectra in a self-supervised manner. Our universal spectral tokenizer processes spectra from a variety of object types and resolutions directly on their native wavelength grids, producing intrinsically aligned, homogeneous, and physically meaningful representations that can be efficiently adapted to achieve competitive performance across a range of downstream tasks. For the first time, we demonstrate that a single model can unify spectral data across resolutions and domains, suggesting that our model can serve as a powerful building block for foundation models in astronomy -- and potentially extend to other scientific domains with heterogeneous sequential data, such as climate and healthcare.

Universal Spectral Tokenization via Self-Supervised Panchromatic Representation Learning

TL;DR

The paper addresses the challenge of unifying heterogeneous, multi-resolution spectral data across astronomical surveys into a single, reusable representation. It introduces a Vision Transformer–based universal spectrum tokenizer that ingests native wavelength grids and learns wavelength-aware embeddings via self-supervised pretraining on SDSS, DESI, GALAH, and APOGEE. Key contributions include a novel architecture that operates without homogenization, a reconstruction objective, and demonstrations that lightweight adapters achieve competitive performance on tasks such as object classification and stellar parameter estimation. The approach enables cross-survey transfer and foundation-model–like scalability for astronomy, with potential extension to other scientific domains featuring irregular sequential data.

Abstract

Sequential scientific data span many resolutions and domains, and unifying them into a common representation is a key step toward developing foundation models for the sciences. Astronomical spectra exemplify this challenge: massive surveys have collected millions of spectra across a wide range of wavelengths and resolutions, yet analyses remain fragmented across spectral domains (e.g., optical vs. infrared) and object types (e.g., stars vs. galaxies), limiting the ability to pool information across datasets. We present a deep learning model that jointly learns from heterogeneous spectra in a self-supervised manner. Our universal spectral tokenizer processes spectra from a variety of object types and resolutions directly on their native wavelength grids, producing intrinsically aligned, homogeneous, and physically meaningful representations that can be efficiently adapted to achieve competitive performance across a range of downstream tasks. For the first time, we demonstrate that a single model can unify spectral data across resolutions and domains, suggesting that our model can serve as a powerful building block for foundation models in astronomy -- and potentially extend to other scientific domains with heterogeneous sequential data, such as climate and healthcare.
Paper Structure (14 sections, 1 equation, 3 figures, 2 tables)

This paper contains 14 sections, 1 equation, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Contributions
  4. A Universal Spectrum Tokenizer
  5. Input processing and encoding
  6. Decoding and reconstruction
  7. Dataset
  8. Training and implementation details
  9. Results
  10. Spectrum reconstruction
  11. Structure of the latent space
  12. Object classification
  13. Physical parameter estimation
  14. Conclusion

Figures (3)

  • Figure 1: Overview of the universal spectrum tokenizer. The encoder creates homogeneous, wavelength-aware embeddings from heterogeneous input spectra; these can be used for a variety of downstream tasks. The decoder reconstructs the input spectrum from the learned representations.
  • Figure 2: Examples of reconstructed spectra from the model for three different object types and four different surveys. Each subfigure represents a different dataset. The top panels show the original spectrum in black and the reconstructed spectrum in red. The bottom panels show the residuals, with raw values in grey and a smoothed version in black. Figures are best viewed zoomed in.
  • Figure 3: UMAP visualization of mean-pooled embeddings for 10,000 SDSS galaxies. The x and y axes are arbitrary. The panels are coloured by stellar mass (left) and redshift (right), and show that the embeddings have strong correlations with these physical properties.