Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Noisy Nonnegative Tucker Decomposition with Sparse Factors and Missing Data

Xiongjun Zhang, Michael K. Ng

TL;DR

This work tackles recovering nonnegative tensors from incomplete, noisy measurements by proposing a sparse nonnegative Tucker decomposition with a maximum-likelihood loss augmented by $\ell_0$ sparsity on factor matrices. The authors derive general error bounds under broad noise models and specialize them to Gaussian, Laplace, and Poisson observations, establishing near-optimal minimax rates. An ADMM-based algorithm is developed to solve the nonconvex, discretization-amenable optimization problem, and the method is validated on synthetic and real data, where it consistently outperforms matrix-based and tensor-tensor product baselines. The results highlight the effectiveness of enforcing sparsity in the factor matrices while maintaining nonnegativity, yielding accurate tensor completion and meaningful latent factors in practical settings.

Abstract

Tensor decomposition is a powerful tool for extracting physically meaningful latent factors from multi-dimensional nonnegative data, and has been an increasing interest in a variety of fields such as image processing, machine learning, and computer vision. In this paper, we propose a sparse nonnegative Tucker decomposition and completion method for the recovery of underlying nonnegative data under noisy observations. Here the underlying nonnegative data tensor is decomposed into a core tensor and several factor matrices with all entries being nonnegative and the factor matrices being sparse. The loss function is derived by the maximum likelihood estimation of the noisy observations, and the $\ell_0$ norm is employed to enhance the sparsity of the factor matrices. We establish the error bound of the estimator of the proposed model under generic noise scenarios, which is then specified to the observations with additive Gaussian noise, additive Laplace noise, and Poisson observations, respectively. Our theoretical results are better than those by existing tensor-based or matrix-based methods. Moreover, the minimax lower bounds are shown to be matched with the derived upper bounds up to logarithmic factors. Numerical examples on both synthetic and real-world data sets demonstrate the superiority of the proposed method for nonnegative tensor data completion.

Noisy Nonnegative Tucker Decomposition with Sparse Factors and Missing Data

TL;DR

This work tackles recovering nonnegative tensors from incomplete, noisy measurements by proposing a sparse nonnegative Tucker decomposition with a maximum-likelihood loss augmented by sparsity on factor matrices. The authors derive general error bounds under broad noise models and specialize them to Gaussian, Laplace, and Poisson observations, establishing near-optimal minimax rates. An ADMM-based algorithm is developed to solve the nonconvex, discretization-amenable optimization problem, and the method is validated on synthetic and real data, where it consistently outperforms matrix-based and tensor-tensor product baselines. The results highlight the effectiveness of enforcing sparsity in the factor matrices while maintaining nonnegativity, yielding accurate tensor completion and meaningful latent factors in practical settings.

Abstract

Tensor decomposition is a powerful tool for extracting physically meaningful latent factors from multi-dimensional nonnegative data, and has been an increasing interest in a variety of fields such as image processing, machine learning, and computer vision. In this paper, we propose a sparse nonnegative Tucker decomposition and completion method for the recovery of underlying nonnegative data under noisy observations. Here the underlying nonnegative data tensor is decomposed into a core tensor and several factor matrices with all entries being nonnegative and the factor matrices being sparse. The loss function is derived by the maximum likelihood estimation of the noisy observations, and the norm is employed to enhance the sparsity of the factor matrices. We establish the error bound of the estimator of the proposed model under generic noise scenarios, which is then specified to the observations with additive Gaussian noise, additive Laplace noise, and Poisson observations, respectively. Our theoretical results are better than those by existing tensor-based or matrix-based methods. Moreover, the minimax lower bounds are shown to be matched with the derived upper bounds up to logarithmic factors. Numerical examples on both synthetic and real-world data sets demonstrate the superiority of the proposed method for nonnegative tensor data completion.
Paper Structure (16 sections, 7 theorems, 135 equations, 4 figures, 1 table, 1 algorithm)

This paper contains 16 sections, 7 theorems, 135 equations, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notation
  4. Multilinear Operators
  5. Kullback-Leibler Divergence and Hellinger Affinity
  6. Sparse NTD and Completion With Noisy Observations
  7. Error Bounds
  8. Additive Gaussian Noise
  9. Additive Laplace Noise
  10. Poisson Observations
  11. Minimax Lower Bounds
  12. ADMM Based Algorithm
  13. Numerical Experiments
  14. Synthetic Data
  15. Image Data
  16. ...and 1 more sections

Key Result

Theorem 4.1

Let the sampling set $\Omega$ be drawn from the independent Bernoulli model with probability $p=\frac{m}{n_1n_2\cdots n_d}$, i.e., $\Omega\sim \textup{Bern}(p)$, and the joint probability density/mass function of $\mathcal{Y}_\Omega$ be defined as (obserPo). For any where $\gamma$ is a constant satisfying then where the expectation is taken with regard to the joint distribution of $\Omega$ and

Figures (4)

  • Figure 1: Relative error versus sampling ratio for synthetic tensors with size $100\times 100\times 100$ and Tucker rank $(5,5,5)$.
  • Figure 2: Plots of MSE on a logarithmic scale versus sampling ratio for synthetic tensors with size $50\times 50\times 50\times 50$ and Tucker rank $(5,5,5,5)$.
  • Figure 3: Relative error versus sampling ratio for the Swimmer dataset.
  • Figure 4: Relative error versus sampling ratio for the COIL-100 dataset.

Theorems & Definitions (26)

  • Remark 3.1
  • Remark 3.2
  • Remark 3.3
  • Remark 3.4
  • Remark 3.5
  • Theorem 4.1
  • Remark 4.1
  • Theorem 4.2
  • Remark 4.2
  • Remark 4.3
  • ...and 16 more