Sparse encoding for more-interpretable feature-selecting representations in probabilistic matrix factorization

TL;DR

This work addresses HPF's deficiency by self-consistently enforcing encoder sparsity, using a generalized additive model (GAM), thereby allowing one to relate each representation coordinate to a subset of the original data features and gains the ability to perform feature selection.

Abstract

Dimensionality reduction methods for count data are critical to a wide range of applications in medical informatics and other fields where model interpretability is paramount. For such data, hierarchical Poisson matrix factorization (HPF) and other sparse probabilistic non-negative matrix factorization (NMF) methods are considered to be interpretable generative models. They consist of sparse transformations for decoding their learned representations into predictions. However, sparsity in representation decoding does not necessarily imply sparsity in the encoding of representations from the original data features. HPF is often incorrectly interpreted in the literature as if it possesses encoder sparsity. The distinction between decoder sparsity and encoder sparsity is subtle but important. Due to the lack of encoder sparsity, HPF does not possess the column-clustering property of classical NMF -- the factor loading matrix does not sufficiently define how each factor is formed from the original features. We address this deficiency by self-consistently enforcing encoder sparsity, using a generalized additive model (GAM), thereby allowing one to relate each representation coordinate to a subset of the original data features. In doing so, the method also gains the ability to perform feature selection. We demonstrate our method on simulated data and give an example of how encoder sparsity is of practical use in a concrete application of representing inpatient comorbidities in Medicare patients.

Figures (8)

• Figure 1: Interpreting hierarchical sparse probabilistic matrix factorization (HPF).(a) Standard HPF: Rates $\lambda_{ui}$ for Poisson-distributed predictions are sparse linear combinations of the learned representation as defined by the decoding matrix; this matrix does not define how representations are derived from the input data. (b) Sparsely-encoded HPF (proposed method): the mapping from input data to representation is given explicitly by a sparse encoding matrix. (c) Interpreting representations: It is tempting but misleading to read the decoding matrices row-wise in determining the feature subsets that contribute to forming a representation coordinate. Representations $\boldsymbol{\theta}_u$ are computed by inferring the statistics of an associated joint posterior distribution $\pi(\ldots\vert \mathbf{Y})$ -- the sets of non-sparse entries in rows of the decoding matrices do not necessarily correspond to feature sets that determine the representation components. However, for sparsely-encoded HPF, the representations are explicit functions of subsets of features.
• Figure 2: Factorization of simulated datasets. The (mean) effective encoding matrix $\mathbf{A} = (\alpha_{ik})$ for each factor process, placed on a common color scale, and the posterior distribution of the background process rate $\varphi_i$ by item for a) Poisson($1$) noise, where there is no relationship between the features, b) linear factor model where every third variable is generated from a dense factor model and the other variables are Poisson($1$) noise, c) nonlinear factor model where every third variable is generated from a dense nonlinear factor model and the other variables are Poisson($1$) noise. See Fig. \ref{['fig:hpfrec_sims']} for standard HPF on these datasets for comparison.
• Figure 3: Decoder matrices $\mathbf{B}=(\beta_{ki})$ for standard HPF factorization of the synthetic datasets of Fig. \ref{['fig:simulation_factorization']} using the python package hpfrec. Shown are posterior means.
• Figure 4: Medicare comorbidity factorization for inpatient visits based on medical claims from a 5% sample of the Medicare Limited Dataset (LDS), in four factor dimensions. Prior to factorization, we mapped each raw ICD diagnostic code into the second tier of the Clinical Classification Software (CCS), counting the number of codes present within each broad category. Shown are posterior means. Left: encoding $\mathbf{A} = (\alpha_{ik})$, middle: decoding $\mathbf{B}^\top = (\beta_{ki})^\top$, right: background $\boldsymbol{\varphi} = ({\varphi}_i)$
• Figure S1: Factorization based on the logarithmic link function of Section \ref{['sec:model']} of the synthetic dataset of Fig. \ref{['fig:simulation_factorization']}

Theorems & Definitions (2)

• Definition 1
• Definition 2