Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Low-Rank Tensor Learning by Generalized Nonconvex Regularization

Sijia Xia, Michael K. Ng, Xiongjun Zhang

TL;DR

A nonconvex model based on transformed tensor nuclear norm for low-rank tensor learning is proposed, and a proximal majorization-minimization (PMM) algorithm is designed to solve the resulting model.

Abstract

In this paper, we study the problem of low-rank tensor learning, where only a few of training samples are observed and the underlying tensor has a low-rank structure. The existing methods are based on the sum of nuclear norms of unfolding matrices of a tensor, which may be suboptimal. In order to explore the low-rankness of the underlying tensor effectively, we propose a nonconvex model based on transformed tensor nuclear norm for low-rank tensor learning. Specifically, a family of nonconvex functions are employed onto the singular values of all frontal slices of a tensor in the transformed domain to characterize the low-rankness of the underlying tensor. An error bound between the stationary point of the nonconvex model and the underlying tensor is established under restricted strong convexity on the loss function (such as least squares loss and logistic regression) and suitable regularity conditions on the nonconvex penalty function. By reformulating the nonconvex function into the difference of two convex functions, a proximal majorization-minimization (PMM) algorithm is designed to solve the resulting model. Then the global convergence and convergence rate of PMM are established under very mild conditions. Numerical experiments are conducted on tensor completion and binary classification to demonstrate the effectiveness of the proposed method over other state-of-the-art methods.

Low-Rank Tensor Learning by Generalized Nonconvex Regularization

TL;DR

A nonconvex model based on transformed tensor nuclear norm for low-rank tensor learning is proposed, and a proximal majorization-minimization (PMM) algorithm is designed to solve the resulting model.

Abstract

In this paper, we study the problem of low-rank tensor learning, where only a few of training samples are observed and the underlying tensor has a low-rank structure. The existing methods are based on the sum of nuclear norms of unfolding matrices of a tensor, which may be suboptimal. In order to explore the low-rankness of the underlying tensor effectively, we propose a nonconvex model based on transformed tensor nuclear norm for low-rank tensor learning. Specifically, a family of nonconvex functions are employed onto the singular values of all frontal slices of a tensor in the transformed domain to characterize the low-rankness of the underlying tensor. An error bound between the stationary point of the nonconvex model and the underlying tensor is established under restricted strong convexity on the loss function (such as least squares loss and logistic regression) and suitable regularity conditions on the nonconvex penalty function. By reformulating the nonconvex function into the difference of two convex functions, a proximal majorization-minimization (PMM) algorithm is designed to solve the resulting model. Then the global convergence and convergence rate of PMM are established under very mild conditions. Numerical experiments are conducted on tensor completion and binary classification to demonstrate the effectiveness of the proposed method over other state-of-the-art methods.

Paper Structure

This paper contains 21 sections, 22 theorems, 216 equations, 6 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Low-Rank Tensor Learning
  3. Tensor SVD
  4. The Contribution
  5. Preliminaries
  6. Nonconvex Model for Low-Rank Tensor Learning
  7. Statistical Guarantee
  8. Optimization Algorithm
  9. PMM Algorithm
  10. Convergence Analysis
  11. ADMM for Solving the Subproblem
  12. Numerical Experiments
  13. Stopping Criterion
  14. Tensor Completion
  15. Parameter Settings
  16. ...and 6 more sections

Key Result

Theorem 1

Kernfeld2015 The transformed tensor singular value decomposition of $\mathcal{X} \in \mathbb {C}^{n_1\times n_2\times n_3}$ is given by $\mathcal{X}= \mathcal{U} \diamond_{\mathbf U}\Sigma\diamond_{\mathbf U}\mathcal{V}^T,$ where $\Sigma \in \mathbb {C}^{n_1\times n_2\times n_3}$ is a diagonal tenso

Figures (6)

  • Figure 1: PSNR values versus index of band of different methods for the Balloons and Lemons datasets. (a) Balloons dataset, where $\textup{SR} = 0.5$ and $\sigma=0.05$. (b) Lemons dataset, where $\textup{SR}= 0.4$ and $\sigma=0.1$.
  • Figure 2: The recovered images (with PSNR values) and zoomed regions of different methods for the 30th band of the Balloons dataset, where $\textup{SR}= 0.4$ and $\sigma=0.05$.
  • Figure 3: The recovered images (with PSNR values) and zoomed regions of different methods for the 30th band of the Lemons dataset, where $\textup{SR}= 0.3$ and $\sigma=0.05$.
  • Figure 4: PSNR values versus index of frame of different methods for the MRI dataset. (a) $\textup{SR} = 0.45$ and $\sigma=0.005$. (b) $\textup{SR}= 0.6$ and $\sigma=0.02$.
  • Figure 5: The recovered images (with PSNR values) and zoomed regions of different methods for the 74th frontal slice of the MRI dataset, where $\textup{SR}= 0.5$ and $\sigma=0.005$.
  • ...and 1 more figures

Theorems & Definitions (36)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Theorem 1
  • Definition 6
  • Remark 1
  • Remark 2
  • Remark 3
  • ...and 26 more