Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Transfer Learning for Missing Not-at-Random Matrix Completion

Akhil Jalan, Yassir Jedra, Arya Mazumdar, Soumendu Sundar Mukherjee, Purnamrita Sarkar

TL;DR

This paper addresses MNAR matrix completion in a transfer-learning setting where the target matrix $Q$ has entire rows and columns missing. It develops a least-squares-based estimator that leverages a noisy source $P$ related to $Q$ via a distribution-shift in latent subspaces, and uses a $G$-optimal design to guide active sampling. Theoretical contributions include minimax lower bounds for both active and passive settings, with the active estimator achieving the lower bound, and a rate-optimal passive estimator under incoherence. Empirical validation on real gene-expression and metabolic-network data demonstrates practical gains of the transfer-learning approach, especially when active sampling is feasible and the source-target pair is coherent.

Abstract

We study transfer learning for matrix completion in a Missing Not-at-Random (MNAR) setting that is motivated by biological problems. The target matrix $Q$ has entire rows and columns missing, making estimation impossible without side information. To address this, we use a noisy and incomplete source matrix $P$, which relates to $Q$ via a feature shift in latent space. We consider both the active and passive sampling of rows and columns. We establish minimax lower bounds for entrywise estimation error in each setting. Our computationally efficient estimation framework achieves this lower bound for the active setting, which leverages the source data to query the most informative rows and columns of $Q$. This avoids the need for incoherence assumptions required for rate optimality in the passive sampling setting. We demonstrate the effectiveness of our approach through comparisons with existing algorithms on real-world biological datasets.

Optimal Transfer Learning for Missing Not-at-Random Matrix Completion

TL;DR

This paper addresses MNAR matrix completion in a transfer-learning setting where the target matrix has entire rows and columns missing. It develops a least-squares-based estimator that leverages a noisy source related to via a distribution-shift in latent subspaces, and uses a -optimal design to guide active sampling. Theoretical contributions include minimax lower bounds for both active and passive settings, with the active estimator achieving the lower bound, and a rate-optimal passive estimator under incoherence. Empirical validation on real gene-expression and metabolic-network data demonstrates practical gains of the transfer-learning approach, especially when active sampling is feasible and the source-target pair is coherent.

Abstract

We study transfer learning for matrix completion in a Missing Not-at-Random (MNAR) setting that is motivated by biological problems. The target matrix has entire rows and columns missing, making estimation impossible without side information. To address this, we use a noisy and incomplete source matrix , which relates to via a feature shift in latent space. We consider both the active and passive sampling of rows and columns. We establish minimax lower bounds for entrywise estimation error in each setting. Our computationally efficient estimation framework achieves this lower bound for the active setting, which leverages the source data to query the most informative rows and columns of . This avoids the need for incoherence assumptions required for rate optimality in the passive sampling setting. We demonstrate the effectiveness of our approach through comparisons with existing algorithms on real-world biological datasets.

Paper Structure

This paper contains 35 sections, 26 theorems, 97 equations, 13 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Setup.
  3. Organization of the Paper
  4. Notation and Problem Setup
  5. Main Findings
  6. Lower Bound for Active Sampling Setting
  7. Estimation Framework
  8. Least Squares Estimator.
  9. Active Sampling.
  10. Passive Sampling
  11. Lower Bound for Passive Sampling
  12. Experiments
  13. Real World Experiments
  14. Simulations
  15. Ablation Studies
  16. ...and 20 more sections

Key Result

Proposition 2.1

Let $m, n \geq 1$ and $d \leq m \land n$. Let $\Psi = (Q, \sigma, p_{\textup{Row}}, p_{\textup{Col}})$ where $Q\in \mathcal{T}_{mn}^{(d)}$, $\sigma^2 > 0$, and $p_{\textup{Row}}, p_{\textup{Col}} \in [0,1]$. Let $\mathop{\mathbb{P}}_\Psi$ denote the law of the random matrix $\widetilde{Q}$ defined a

Figures (13)

  • Figure 1: The missingness matrix for gene expression levels on Day $2$ of a sepsis study parnell2013identifying shows entire rows (patients) and columns (genes) as missing, due to e.g. probe-target hybridization failure of the Illumina HT-12 gene expression microarray hu2021next. We mark missing entries as $0$ (white) and present entries as $1$ (blue). This motivates our missingness model (Eq. \ref{['eq:active_sampling']} and Eq. \ref{['eq:passive_sampling']}).
  • Figure 2: Max-squared error of $\hat{Q} - Q$. Here, $\widetilde{Q}$ has $p_{\textup{Row}} = p_{\textup{Col}}$ varying along the $x$-axis, which displays $p_{\textup{Row}}^2$. We set $\sigma_Q = 0.1$, and $P$ is fully observed. For each method, we show the median of the errors across $50$ independent runs, as well as the $[10, 90]$ percentile.
  • Figure 3: Max-squared error of $\hat{Q} - Q$, with the same experimental parameters as Figure \ref{['fig:rnaseq_comparison_abs_error']}.
  • Figure 4: Ablation study for the effect of additive target noise in the Matrix Partition Model. For each method, we display the median max-squared error across $10$ independent runs, as well as the $[10, 90]$ percentile.
  • Figure 5: We test the effect of growing the target additive noise parameter $\sigma_Q$.
  • ...and 8 more figures

Theorems & Definitions (45)

  • Definition 1.1: Incoherence
  • Definition 1.2: Matrix Transfer Model
  • Proposition 2.1: Minimax Error of MNAR Matrix Completion Without Transfer
  • Theorem 2.2: Minimax Lower Bound for $Q$-estimation with Active Sampling
  • Definition 2.3: $\epsilon$-approximate $G$-optimal design
  • Proposition 2.4: Tensorization of $G$-optimal design
  • Theorem 2.6: Generic error bound for active sampling
  • Remark 2.7: Minimax Optimality for MNAR and MCAR Source Data
  • Remark 2.8: Incoherence-free minimax optimality
  • Theorem 2.9: Generic Error Bound for $\hat{Q}$
  • ...and 35 more