Minuet: A Diffusion Autoencoder for Compact Semantic Compression of Multi-Band Galaxy Images

TL;DR

Minuet tackles the challenge of learning compact, interpretable representations from large multi-band galaxy images by combining a transformer-based encoder with a diffusion autoencoder decoder to compress grz images into five latent features. A conditional neural-spline flow maps these latent features to posterior distributions of redshift, stellar mass, and star formation rate, enabling fast, probabilistic inferences competitive with traditional SED fitting. The study demonstrates strong correlations between the five latent features and Galaxy Zoo morphological labels, and shows that the latent space supports meaningful similarity searches, with the method remaining robust to orientation. The results suggest a practical path toward scalable, interpretable feature extraction for LSST-era datasets, with potential applications in time-domain host-galaxy analysis and anomaly detection.

Abstract

The Vera C. Rubin Observatory is slated to observe nearly 20 billion galaxies during its decade-long Legacy Survey of Space and Time. The rich imaging data it collects will be an invaluable resource for probing galaxy evolution across cosmic time, characterizing the host galaxies of transient phenomena, and identifying novel populations of anomalous systems. While machine learning models have shown promise for extracting galaxy features from multi-band astronomical imaging, the large dimensionality of the learned latent space presents a challenge for mechanistic interpretability studies. In this work, we present Minuet, a low-dimensional diffusion autoencoder for multi-band galaxy imaging. Minuet is trained to reconstruct 72x72-pixel $grz$ image cutouts of 6M galaxies within $z<1$ from the Dark Energy Camera Legacy Survey using only five latent dimensions. By using a diffusion model conditioned on the transformer-based autoencoder's output for image reconstruction, we achieve semantically-meaningful latent representations of galaxy images while still allowing for high-fidelity, probabilistic reconstructions. We train a series of binary classifiers on Minuet's latent features to quantify their connection to morphological labels from Galaxy Zoo, and a conditional flow to produce posterior distributions of SED-derived redshifts, stellar masses, and star-formation rates. We further show the value of Minuet for nearest neighbor searches in the learned latent space. Minuet provides strong evidence for the low intrinsic dimensionality of galaxy imaging, and introduces a class of astrophysical models that produce highly compact representations for diverse science goals.

Figures (15)

• Figure 1: Distribution of redshift (top) and stellar mass (bottom) for galaxies in the training (green) and testing (orange) sets. The validation set is identical in size and distribution to the test set.
• Figure 2: The Minuet architecture. The encoder consists of four Perceiver encoder layers, in which the input tokens as key and value attend to a bottlenecked representation of the input as query. Image inputs have dimensionality $B \times C \times H \times W$, where $C=3$ is the channel dimension, $B=128$ is the batch size, $W=72$ is the image width and $H=72$ is the image height. The intermediate output is a tensor of dimensionality $L\times M$, which is projected to a final output of $L\times1$ latent tokens. In the decoder stage, the latent tokens are concatenated to a learnable embedding of the diffusion time and passed through the same Perceiver modules as the encoder, before finally being projected to the input dimensionality.
• Figure 3: 20 DECaLS $grz$-band images of galaxies in our bright ($r<18$) galaxy sub-sample (top row), segmentation masks (middle row), and sample reconstructions from Minuet using 5 latent features (bottom row). Key galaxy color and brightness profile features are preserved, although some high-frequency information is lost and the reconstructions do not preserve galaxy orientation.
• Figure 4: Left Column: Ground truth images of three galaxies from our bright test set. Middle Columns: Five samples drawn from the learned posterior using DDIM starting from random Gaussian noise realizations (middle five columns). Image samples are agnostic to galaxy orientation (as seen in middle and bottom rows) and neighboring objects due to the segmentation mask applied in training.Right Column: Radial frequency distributions in $grz$ (green, red, and dark red, respectively). Ground truth distributions are shown as dashed lines, and mean and shaded regions correspond to the 1$\sigma$ standard deviation across the five reconstructions.
• Figure 5: Corner plot of all latent features for the sample of galaxies cross-matched with the catalog from 2022Zou_SED, and colored by the best redshift for the galaxy (spectroscopic if available, else the photometric value inferred in 2022Zou_SED is used).