Multi-step Inertial Accelerated Doubly Stochastic Gradient Methods for Block Term Tensor Decomposition

TL;DR

The paper addresses efficient optimization for a composite nonconvex-nonsmooth objective arising in rank-$(L_r,L_r,1)$ block-term tensor decomposition. It introduces Midas-LL1, a multi-step inertial accelerated doubly stochastic gradient method with an extended variance-reduced estimator, and provides subsequential and sequential convergence guarantees with an $O(\varepsilon^{-2})$ iteration bound to reach an $\varepsilon$-stationary point. The method leverages fiber-sampling, partition-wise stochastic gradients, and a novel Lyapunov analysis to ensure convergence, while empirical results on hyperspectral and video datasets demonstrate faster convergence and improved reconstruction quality over state-of-the-art baselines such as MVNTF, especially with three-step acceleration and variance reduction. These results highlight the practical impact of combining multi-step inertial acceleration with variance-reduced stochastic gradients for large-scale, structured tensor decompositions in real-world data.

Abstract

In this paper, we explore a specific optimization problem that combines a differentiable nonconvex function with a nondifferentiable function for multi-block variables, which is particularly relevant to tackle the multilinear rank-($L_r$,$L_r$,1) block-term tensor decomposition model with a regularization term. While existing algorithms often suffer from high per-iteration complexity and slow convergence, this paper employs a unified multi-step inertial accelerated doubly stochastic gradient descent method tailored for structured rank-$\left(L_r, L_r, 1\right)$ tensor decomposition, referred to as Midas-LL1. We also introduce an extended multi-step variance-reduced stochastic estimator framework. Our analysis under this new framework demonstrates the subsequential and sequential convergence of the proposed algorithm under certain conditions and illustrates the sublinear convergence rate of the subsequence, showing that the Midas-LL1 algorithm requires at most $\mathcal{O}(\varepsilon^{-2})$ iterations in expectation to reach an $\varepsilon$-stationary point. The proposed algorithm is evaluated on several datasets, and the results indicate that Midas-LL1 outperforms existing state-of-the-art algorithms in terms of both computational speed and solution quality.

Figures (9)

• Figure 1: BTD for a third-order tensor.
• Figure 2: BTD with rank-$\left(L_r, L_r, 1\right)$ for a third-order tensor.
• Figure 3: The illustration of multi-step inertial acceleration.
• Figure 4: Fiber-sampling strategy with batchsize $\left|\mathcal{F}_{n}\right|=3$.
• Figure 5: Numerical experiments on the $80 \times 84 \times 204$ Salinas dataset with different values of $R$ and $L$.

Theorems & Definitions (21)

• Definition 1
• Proposition 1
• proof
• Definition 2
• Remark 1
• Remark 2
• Definition 3
• Lemma 1
• proof
• Lemma 2