On the Relation Between Autoencoders and Non-negative Matrix Factorization, and Their Application for Mutational Signature Extraction

TL;DR

This work establishes a theoretical and empirical link between non-negative autoencoders (AE) and convex NMF (C-NMF) by showing that a shallow, non-negative AE can be equivalent to C-NMF when key components are fixed and activations are identities. It then compares NMF and AE-NMF on mutational signature extraction from cancer genomes, finding that NMF provides better reconstruction while both methods yield highly consistent signatures with strong COSMIC alignment; thus AE-NMF does not outperform NMF in this application. The study highlights practical implications: the equivalence clarifies parameter interpretation for AE-NMF, and while AE-NMF can be faster or useful for modeling nonlinearity, it does not improve mutational signature extraction under the settings tested. The results emphasize the importance of methodological choices (loss functions, data orientation, optimization) when substituting NMF with autoencoder-based approaches.

Abstract

The aim of this study is to provide a foundation to understand the relationship between non-negative matrix factorization (NMF) and non-negative autoencoders enabling proper interpretation and understanding of autoencoder-based alternatives to NMF. Since its introduction, NMF has been a popular tool for extracting interpretable, low-dimensional representations of high-dimensional data. However, recently, several studies have proposed to replace NMF with autoencoders. This increasing popularity of autoencoders warrants an investigation on whether this replacement is in general valid and reasonable. Moreover, the exact relationship between non-negative autoencoders and NMF has not been thoroughly explored. Thus, a main aim of this study is to investigate in detail the relationship between non-negative autoencoders and NMF. We find that the connection between the two models can be established through convex NMF, which is a restricted case of NMF. In particular, convex NMF is a special case of an autoencoder. The performance of NMF and autoencoders is compared within the context of extraction of mutational signatures from cancer genomics data. We find that the reconstructions based on NMF are more accurate compared to autoencoders, while the signatures extracted using both methods show comparable consistencies and values when externally validated. These findings suggest that the non-negative autoencoders investigated in this article do not provide an improvement of NMF in the field of mutational signature extraction.

Figures (5)

• Figure 1: Schematic representation of the composition of basis vectors and weights in NMF (top) and C-NMF and AE-NMF (bottom).
• Figure 2: Estimated and true weights and basis vectors of the simulated data. Note the light green lines representing C-NMF coincide with the the dashed, coral lines of AE-NMF since the estimates found by both methods are almost identical.
• Figure 3: Second principal component plotted agains the first principal component of the de novo extracted signatures from the 30 train/test splits for AE-NMF and NMF (columns) and each diagnosis (rows). Points are colored by the PAM clustering assignment, and the cluster mediod is highlighted with a black outline. NMF: non-negative matrix factorization; AE: autoencoder; PAM: partition around mediods.
• Figure 4: Boxplots of the training and test errors of 30 train/test splits of the ovary, prostate and uterus cohorts. NMF and AE-NMF errors resulting from the same splits are connected by a black line. The boxes are colored corresponding to the method used, and a green diamond depicts the average error. The y-axis is on log 10 scale.
• Figure 5: Boxplots of the ACS between each combination of signatures, extracted using NMF or AE-NMF and across the 30 splits of the data matrix for the ovary, prostate, and uterus cohort. The average is marked by a green diamond.