Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust low-rank tensor regression via clipping and Huber loss

Kangqiang Li, Bingqi Liu, Yang Yang, Li Wang

TL;DR

The paper develops a robust low-rank tensor regression framework that handles heavy-tailed noise by combining truncation (clipping) and Huber loss. It formulates a Tucker-structured low-rank estimator updated via robust gradient descent and proves minimax-optimal convergence rates and optimal sample complexity under finite second moments, with generalizations to asymmetric robust losses. Through simulations and real-data applications in image recovery and Beijing air quality, the method demonstrates superior stability and statistical performance over traditional least-squares approaches, including a phase-transition behavior in convergence. The work also outlines practical extensions to higher-order tensors and discusses data-driven tuning and relaxations of standard assumptions for future study.

Abstract

In this paper, we construct a parameter estimation framework for robust low-rank tensor regression based on a truncation method and Huber loss, specifically focusing on models with random noise having only finite second-order moments. Through a robust gradient descent method, our proposed Huber-type estimator is theoretically optimal in two aspects: (1) its statistical error rate matches the optimal upper bound established for the traditional least squares method under sub-Gaussian error; and (2) the sample complexity for recovering the tensor parameter is also optimal. Extensive numerical experiments demonstrate the robustness of our estimator, indicating that the utilization of truncation and Huber loss significantly enhances stability and statistical effectiveness, outperforming the traditional least squares method. Additionally, the phenomenon of phase transition in the convergence rate of the proposed estimator is confirmed through simulation. Furthermore, applications to image recovery and the Beijing air-quality dataset demonstrate the practical effectiveness of our method.

Robust low-rank tensor regression via clipping and Huber loss

TL;DR

The paper develops a robust low-rank tensor regression framework that handles heavy-tailed noise by combining truncation (clipping) and Huber loss. It formulates a Tucker-structured low-rank estimator updated via robust gradient descent and proves minimax-optimal convergence rates and optimal sample complexity under finite second moments, with generalizations to asymmetric robust losses. Through simulations and real-data applications in image recovery and Beijing air quality, the method demonstrates superior stability and statistical performance over traditional least-squares approaches, including a phase-transition behavior in convergence. The work also outlines practical extensions to higher-order tensors and discusses data-driven tuning and relaxations of standard assumptions for future study.

Abstract

In this paper, we construct a parameter estimation framework for robust low-rank tensor regression based on a truncation method and Huber loss, specifically focusing on models with random noise having only finite second-order moments. Through a robust gradient descent method, our proposed Huber-type estimator is theoretically optimal in two aspects: (1) its statistical error rate matches the optimal upper bound established for the traditional least squares method under sub-Gaussian error; and (2) the sample complexity for recovering the tensor parameter is also optimal. Extensive numerical experiments demonstrate the robustness of our estimator, indicating that the utilization of truncation and Huber loss significantly enhances stability and statistical effectiveness, outperforming the traditional least squares method. Additionally, the phenomenon of phase transition in the convergence rate of the proposed estimator is confirmed through simulation. Furthermore, applications to image recovery and the Beijing air-quality dataset demonstrate the practical effectiveness of our method.
Paper Structure (13 sections, 6 theorems, 62 equations, 8 figures, 4 tables, 2 algorithms)

This paper contains 13 sections, 6 theorems, 62 equations, 8 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Notation and definitions
  3. Main result
  4. Model and theoretical results
  5. Generalization of the loss function
  6. Simulation analysis and application
  7. Numerical experiments
  8. Phase transition phenomenon
  9. Image recovery
  10. An application to Beijing air-quality dataset
  11. Discussion
  12. Proof of Theorem \ref{['theorem1']}
  13. Proof of Theorem \ref{['theorem2']}

Key Result

Theorem 1

Assume that the following conditions hold: If $\varpi\asymp_{K,k}\left(Mn/df\right)^{\frac{1}{2}}, \tau\asymp_{K,k}\left(Rn/df\right)^{\frac{1}{2}}$, $b \asymp \bar{\lambda}^{1 / 4}$ and $a\asymp\frac{\bar{\lambda}}{\kappa^{2}}$ are chosen, then for $\forall t> \log(13)$, there exist positive constants $c_{0},c_{1},c_{2}$ and $\{C_{i}\}_{i=1 with probability at least $1-6\exp(-c_{1}\bar{p})-4\exp

Figures (8)

  • Figure 1: Tucker decomposition of tensor $\mathcal{A}=\llbracket \mathcal{S} ; \mathbf{U}_{1}, \mathbf{U}_{2},\mathbf{U}_{3} \rrbracket\in \mathbb{R}^{p_{1}\times p_{2}\times p_{3}}$.
  • Figure 2: Illustration of Huber loss, square loss and truncation function.
  • Figure 3: Comparison of statistical performance between RGD and GD. $T$ and the $\text{error}$ represent the number of iterations and $\left\|\mathcal{A}^{(T)}-\mathcal{A}^{\star}\right\|_{F}$ respectively.
  • Figure 4: The trend of statistical errors with varying $\delta$ where $T_{\max}=500$.
  • Figure 5: Images constructed by randomly selecting an experimental result from Table \ref{['tab4']}.
  • ...and 3 more figures

Theorems & Definitions (16)

  • Definition 1: Tensor Tucker decomposition
  • Definition 2: Huber loss Huber1964
  • Definition 3: Clipping function
  • Theorem 1
  • Remark 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Corollary 1
  • Theorem 2
  • ...and 6 more