Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptively-weighted Nearest Neighbors for Matrix Completion

Tathagata Sadhukhan, Manit Paul, Raaz Dwivedi

TL;DR

AWNN tackles matrix completion with missing data by adaptively weighting nearest-neighbor rows and solving a convex optimization to determine the weights, jointly estimating the noise variance via a fixed-point update. This yields data-driven, hyperparameter-free weight selection and provable finite-sample guarantees without assuming low-rank structure. In simulations, AWNN outperforms standard baselines and approaches an oracle that knows the noise variance, demonstrating robust, fast, and parameter-free performance under MCAR and fully observed settings. The approach holds practical significance for recommender systems and causal inference tasks where missing data and model flexibility are prevalent.

Abstract

In this technical note, we introduce and analyze AWNN: an adaptively weighted nearest neighbor method for performing matrix completion. Nearest neighbor (NN) methods are widely used in missing data problems across multiple disciplines such as in recommender systems and for performing counterfactual inference in panel data settings. Prior works have shown that in addition to being very intuitive and easy to implement, NN methods enjoy nice theoretical guarantees. However, the performance of majority of the NN methods rely on the appropriate choice of the radii and the weights assigned to each member in the nearest neighbor set and despite several works on nearest neighbor methods in the past two decades, there does not exist a systematic approach of choosing the radii and the weights without relying on methods like cross-validation. AWNN addresses this challenge by judiciously balancing the bias variance trade off inherent in weighted nearest-neighbor regression. We provide theoretical guarantees for the proposed method under minimal assumptions and support the theory via synthetic experiments.

Adaptively-weighted Nearest Neighbors for Matrix Completion

TL;DR

AWNN tackles matrix completion with missing data by adaptively weighting nearest-neighbor rows and solving a convex optimization to determine the weights, jointly estimating the noise variance via a fixed-point update. This yields data-driven, hyperparameter-free weight selection and provable finite-sample guarantees without assuming low-rank structure. In simulations, AWNN outperforms standard baselines and approaches an oracle that knows the noise variance, demonstrating robust, fast, and parameter-free performance under MCAR and fully observed settings. The approach holds practical significance for recommender systems and causal inference tasks where missing data and model flexibility are prevalent.

Abstract

In this technical note, we introduce and analyze AWNN: an adaptively weighted nearest neighbor method for performing matrix completion. Nearest neighbor (NN) methods are widely used in missing data problems across multiple disciplines such as in recommender systems and for performing counterfactual inference in panel data settings. Prior works have shown that in addition to being very intuitive and easy to implement, NN methods enjoy nice theoretical guarantees. However, the performance of majority of the NN methods rely on the appropriate choice of the radii and the weights assigned to each member in the nearest neighbor set and despite several works on nearest neighbor methods in the past two decades, there does not exist a systematic approach of choosing the radii and the weights without relying on methods like cross-validation. AWNN addresses this challenge by judiciously balancing the bias variance trade off inherent in weighted nearest-neighbor regression. We provide theoretical guarantees for the proposed method under minimal assumptions and support the theory via synthetic experiments.
Paper Structure (20 sections, 9 theorems, 76 equations, 2 figures)

This paper contains 20 sections, 9 theorems, 76 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Our Contributions
  3. Related Work
  4. Organization
  5. Problem set-up and Motivation
  6. Background and early insights
  7. Algorithm and Theoretical guarantees
  8. Matrix denoising
  9. Matrix completion
  10. Simulation Study
  11. Baselines
  12. Experimental setup
  13. Results
  14. Proof of \ref{['thm:row_wise_bounds']}
  15. Proof of \ref{['thm:optimal_weights']}
  16. ...and 5 more sections

Key Result

Proposition 1

Let $\widehat{\theta}_{i,j} = \sum_{i' = 1}^n w_{i'} X_{i',j}$ where $w^{(i)} = ( w_{1}, \cdots, w_{n})$ is the weight vector associated with the $i$-th row. Under assumptions assump:dist_concentration and assump:sub_g_noise, the row-wise mean squared error for the $i$-th row ($i \in [n]$) has the

Figures (2)

  • Figure 1: MSE of $\mathsf{AWNN}$ and the benchmarks as a function of number of rows $n(=m)$. Results are averaged across 10 runs where signals were generated from Lipschitz function ($f$ with $\lambda=1$) with latent variables' dimension $d = 2$ and $\mathsf{SNR}=1$.
  • Figure 2: Variation of $\mathsf{AWNN}$'s MSE behaviour with changing smoothness levels $(\lambda)$ of signals $\mathopen{}\mathclose{\left\{ \theta_{i,j} \right \}_{i,j\in[n]\times[m]}}$ in synthetic data. Results are averaged across 10 runs where signals were generated from Lipschitz function ($f$ with $\lambda\in\mathopen{}\mathclose{\left\{ 0.5, 0.75, 1 \right \}$) with latent variables' dimension $d = 2$. Top and bottom row correspond to MCAR and no missingness setup respectively while left and right column correspond to $\mathsf{SNR}}$ of 10 and 2 respectively.

Theorems & Definitions (10)

  • Proposition 1
  • Proposition 2
  • Remark 1
  • Theorem 1
  • Lemma 1
  • Corollary 1
  • Lemma 2
  • Theorem 2
  • Corollary 2
  • Lemma 3