Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Finding Rule-Interpretable Non-Negative Data Representation

Matej Mihelčić, Pauli Miettinen

TL;DR

The paper addresses the interpretability gap in non-negative matrix factorization by introducing a rule-describable NMF (DNMF) framework that constrains latent factors to be described by subsets of input rules. It develops three optimization schemes (\textit{DNMF_ind}, \textit{DNMF_dir}, and \textit{DNMF_comb}) and a novel rule-clustering algorithm to map rules to factors, with multiplicative update rules ensuring convergence. Across fifteen datasets and multiple rule types, DNMF_dir often achieves higher factor–rule correspondence and descriptive clarity, while DNMF_ind preserves stable reconstruction quality; together with DNMF_comb, they demonstrate improved interpretability and data fusion capabilities. A gene-function use-case demonstrates the practical value of multi-source data fusion, where DNMF_dir enhances predictive performance and yields interpretable, rule-based embeddings for downstream analyses.

Abstract

Non-negative Matrix Factorization (NMF) is an intensively used technique for obtaining parts-based, lower dimensional and non-negative representation. Researchers in biology, medicine, pharmacy and other fields often prefer NMF over other dimensionality reduction approaches (such as PCA) because the non-negativity of the approach naturally fits the characteristics of the domain problem and its results are easier to analyze and understand. Despite these advantages, obtaining exact characterization and interpretation of the NMF's latent factors can still be difficult due to their numerical nature. Rule-based approaches, such as rule mining, conceptual clustering, subgroup discovery and redescription mining, are often considered more interpretable but lack lower-dimensional representation of the data. We present a version of the NMF approach that merges rule-based descriptions with advantages of part-based representation offered by the NMF. Given the numerical input data with non-negative entries and a set of rules with high entity coverage, the approach creates the lower-dimensional non-negative representation of the input data in such a way that its factors are described by the appropriate subset of the input rules. In addition to revealing important attributes for latent factors, their interaction and value ranges, this approach allows performing focused embedding potentially using multiple overlapping target labels.

Finding Rule-Interpretable Non-Negative Data Representation

TL;DR

The paper addresses the interpretability gap in non-negative matrix factorization by introducing a rule-describable NMF (DNMF) framework that constrains latent factors to be described by subsets of input rules. It develops three optimization schemes (\textit{DNMF_ind}, \textit{DNMF_dir}, and \textit{DNMF_comb}) and a novel rule-clustering algorithm to map rules to factors, with multiplicative update rules ensuring convergence. Across fifteen datasets and multiple rule types, DNMF_dir often achieves higher factor–rule correspondence and descriptive clarity, while DNMF_ind preserves stable reconstruction quality; together with DNMF_comb, they demonstrate improved interpretability and data fusion capabilities. A gene-function use-case demonstrates the practical value of multi-source data fusion, where DNMF_dir enhances predictive performance and yields interpretable, rule-based embeddings for downstream analyses.

Abstract

Non-negative Matrix Factorization (NMF) is an intensively used technique for obtaining parts-based, lower dimensional and non-negative representation. Researchers in biology, medicine, pharmacy and other fields often prefer NMF over other dimensionality reduction approaches (such as PCA) because the non-negativity of the approach naturally fits the characteristics of the domain problem and its results are easier to analyze and understand. Despite these advantages, obtaining exact characterization and interpretation of the NMF's latent factors can still be difficult due to their numerical nature. Rule-based approaches, such as rule mining, conceptual clustering, subgroup discovery and redescription mining, are often considered more interpretable but lack lower-dimensional representation of the data. We present a version of the NMF approach that merges rule-based descriptions with advantages of part-based representation offered by the NMF. Given the numerical input data with non-negative entries and a set of rules with high entity coverage, the approach creates the lower-dimensional non-negative representation of the input data in such a way that its factors are described by the appropriate subset of the input rules. In addition to revealing important attributes for latent factors, their interaction and value ranges, this approach allows performing focused embedding potentially using multiple overlapping target labels.
Paper Structure (12 sections, 2 theorems, 11 equations, 3 figures, 7 tables, 1 algorithm)

This paper contains 12 sections, 2 theorems, 11 equations, 3 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. NMF with describable latent factors
  4. Algorithms
  5. Novel algorithm for rule clustering and assignment to NMF factors
  6. Experiments
  7. Experimental setup
  8. Results
  9. Example factor descriptions
  10. Related Work
  11. Use case
  12. Conclusions

Key Result

Theorem 1

The function $J$ of eq:J is monotonically decreasing under the update rule eq:upF, for any constant $\varepsilon>0,\ \mathbf{F}\geq \varepsilon,\ \mathbf{G}\geq \varepsilon$. Every limit point obtained using multiplicative updates (eq:upF) is a stationary point of the strengthened optimization probl

Figures (3)

  • Figure 1: Factor correspondence accuracy achieved by the $\mathit{DNMF}_{\text{ind}}$, $\mathit{DNMF}_{\text{dir}}$, $\mathit{NMF}_{\text{MU}}$ and $\mathit{NMF}_{\text{sp}}$ approaches on three different datasets using supervised rules (above) and descriptive rules (below).
  • Figure 2: Relation between factor $f_1$ and the rules $r_1,\ r_2$ for the $\mathit{DNMF}_{\text{ind}}$ method (left) and the $\mathit{DNMF}_{\text{dir}}$ method (right).
  • Figure 3: Number of improved functions across $10$ different runs compared to phyletic profile approach by $\mathit{NMF}_{\text{MU}}$ (bottom) and $\mathit{DNMF}_{\text{dir}}$ (top).

Theorems & Definitions (7)

  • Definition 1
  • Definition 2
  • Definition 3
  • Theorem 1
  • proof
  • Theorem 2
  • proof