Deciphering galaxy images using machine vision -- Combining variational autoencoder and principal component analysis for feature extraction

TL;DR

The paper addresses decoding galaxy morphology from images using an unsupervised feature-extraction pipeline that combines a variational autoencoder (VAE) with principal component analysis (PCA). It demonstrates that a 256×256 gri representation of EAGLE mock galaxies can be compressed into 35 latent features, from which 10–12 principal components capture 99.9% of the variance, with 11 PCs typically sufficing; the first PCs predominantly encode galaxy size via the half-light radius $R_{1/2}$. PCA reduces entanglement between latent features and traditional structural measures (e.g., the half-light radius and Sérsic index) from roughly 14.5 and 6 latent features to about 2 PCs, and UMAP analyses show clear neighborhood preservation and separations among morphological classes. Overall, the approach provides data-driven, scalable descriptors that complement conventional structural analyses and can identify atypical galaxy populations in large imaging surveys, with implications for classification, clustering, and anomaly detection on real data.

Abstract

Here, we present a machine vision approach, combining a VAE framework with PCA, to decipher galaxy images. Using mock gri-band images from the EAGLE simulation, the VAE finds that around 35 features are needed to describe the images. Adding the PCA, we identify an optimal range of 10-12 features needed to capture 99.9% of the variance in galaxy images. The exact optimal number varies with galaxy complexity: disk-dominated galaxies require 12 features, bulge-dominated galaxies need 9, and intermediate systems require 10-11 features. Correlations between extracted PCA features and structural measurements reveal that the VAE prioritizes galaxy size during reconstruction, with half-light radius strongly correlating with the highest-ranked principal components. Subsequent features capture morphology-dependent characteristics: disk-dominated galaxies emphasize size, asymmetry, and position angle; bulge-dominated systems focus on size, concentration, and axis ratio; while intermediate galaxies show enhanced attention to Sersic index, indicating greater emphasis on accurately reproducing both disk and bulge structures. The PCA process significantly reduces the entanglement of the features compared to the raw VAE latent features, decreasing the correlations with the half-light radius and the Sersic index from 14.5+-1.0 and 6.0+-1.5 features, respectively, to only 2.0+-1.0 components after PCA. Using UMAP, we construct two-dimensional visualizations that preserve neighborhood relationships from the high-dimensional feature space. This demonstrates that machine vision can effectively distinguish galaxy populations across different morphological types, including systems with atypical structures that may be overlooked by traditional classification methods, providing a data-driven complement to conventional structural measurements.

Figures (13)

• Figure 1: A schematic architecture of the VAE used in the feature learning process. The orange blocks in the intermediate layers represent the latent features, sampled using the means and log-variances.
• Figure 2: The performance of the VAE as a function of the number of latent features retained. The metrics are the residual between the input and reconstructed images (top), the KL divergence (middle), and the number of principal components (PCs) explaining 99.9 per cent of the variance in the data (bottom). Each black dot represents the mean values of three runs for a specific number of latent features, while red dots correspond to cases with 25 runs. In each run the initial weights for the neural networks are chosen randomly. Error bars indicate the uncertainty, defined by the minimum and maximum values of the metric calculated across multiple runs.
• Figure 3: Examples of galaxy images grouped by morphological type: from left to right disk-dominated ($D/T>0.2$), intermediate ($0.1\leq{D/T}\leq 0.2$), and bulge-dominated ($D/T<0.1$). Above each panel, the $D/T$ value is provided for each galaxy. The top and second rows show the original and reconstructed images, respectively. The third row presents the residuals between the original and reconstructed images, and the residual values are provided at the corner. The bottom row displays feature weight maps, highlighting the regions that contribute most to the reconstruction with the reddest colour shading.
• Figure 4: Distributions of the mean value ($\mu$) in the 35 latent feature vectors extracted by the VAE shown by the histograms. Above each panel, the mean (left) and standard deviation (right) of the corresponding feature's mean ($\mu$) values are shown in the parentheses. The solid black line in each panel shows the Gaussian distribution plotted using the calculated mean and standard deviation values.
• Figure 5: Example illustration of galaxy appearance transitions resulting from variations in individual latent features. In each row, a single latent feature is varied between the maximum and minimum values of this feature across the dataset, while all others are fixed at their mean values of all samples. The top row shows a feature with low correlation ($\mathrm{dCor}<0.2$) to any structural measurement. The second and third rows present features with the strongest correlations to Sérsic index and half-light radius, respectively.