On Trimming Tensor-structured Measurements and Efficient Low-rank Tensor Recovery

TL;DR

The paper addresses the challenge of recovering low-rank tensors from memory-efficient, tensor-structured measurements that typically lack TensorRIP guarantees. It introduces local trimming to restore geometry and enable TIHT-based recovery, and proposes two algorithms—TrimTIHT and KaczTIHT—that leverage adaptive trimming and randomized Kaczmarz steps, respectively. The authors provide initial convergence guarantees and demonstrate through synthetic and real-data experiments that these methods outperform standard TensorIHT, especially for face-splitting measurements and higher tensor ranks. This work enables efficient, scalable tensor recovery from memory-friendly sketches with practical impact on high-dimensional data processing tasks that rely on tensor structure.

Abstract

In this paper, we take a step towards developing efficient hard thresholding methods for low-rank tensor recovery from memory-efficient linear measurements with tensorial structure. Theoretical guarantees for many standard iterative low-rank recovery methods, such as iterative hard thresholding (IHT), are based on model assumptions on the measurement operator, like the restricted isometry property (RIP). However, tensor-structured random linear maps -- while memory-efficient and convenient to apply -- lack good restricted isometry properties; that is, they do not preserve the norms of low-rank tensors sufficiently well. To address this, we propose local trimming techniques that provably restore point-wise geometry-preservation properties of tensor-structured maps, making them comparable to those of unstructured sub-Gaussian measurements. Then, we propose two novel versions of tensor IHT algorithms: an adaptive gradient trimming algorithm and a randomized Kaczmarz-based IHT algorithm, that efficiently recover low-rank tensors from linear measurements. We provide initial theoretical guarantees for the proposed methods and present numerical experiments on real and synthetic data, highlighting their efficiency over the original TensorIHT for low HOSVD and CP-rank tensors.

Figures (5)

• Figure 3: Recovery from Gaussian measurements: Fraction of $50$ artificially generated random tensors of a certain rank, recovered successfully using TIHT, TrimTIHT and KaczTIHT at different compression levels. The proposed methods require less measurements for successful recovery, especially for the higher ranks.
• Figure 4: Recovery from Gaussian measurements: Relative error dynamic during the recovery with TIHT, KaczTIHT and TrimTIHT of the tensors of different low rank. KaczTIHT converges faster and to a lower error, especially for higher rank tensors, and TIHT starts diverging. The lines correspond to the median over $40$ sample runs, and the band represents the inter-quartile range.
• Figure 5: Recovery from face-splitting product of Gaussian measurements: Fraction of $50$ artificially generated random tensors of a certain rank, recovered successfully using TIHT, TrimTIHT and KaczTIHT from different levels of compression. The proposed methods enable successful recovery from memory-efficient measurements that is not achieved by TIHT.
• Figure 6: Recovery from face-splitting product of Gaussian measurements: Relative error dynamic during the recovery with KaczTIHT and TrimTIHT of the tensors of different low rank. KaczTIHT converges faster and to a lower error, especially for higher rank tensors. The lines correspond to the median over $40$ sample runs, and the band represents the interquartile range.
• Figure 7: (Left) Relative error of recovered Candle video dataset using KaczTIHT and TrimTIHT from Gaussian and face-splitting measurements. The tensor is of relatively high rank $(9,8,5)$, the compression is to $30\%$ of its size. TIHT method immediately diverges on the same setup. The achieved relative error is compatible with $0.009$ HOSVD fitting error. (Right) The first frame of the original video and its recovered versions are presented in the first row, and the last frame and its recovered images are depicted in the bottom row.

Theorems & Definitions (43)

• Theorem 1: Theorem \ref{['theorem:trim_trip']} and Proposition \ref{['prop:db_friendly_trip']}, informal version
• Theorem 2: Low CP Rank Approximation song2019relative
• Theorem 3: Low HOSVD Rank Approximation song2019relative
• Definition 2.1
• Definition 2.2: TensorRIP
• Definition 2.3: $\varepsilon$-net and Covering Number
• Remark 1: Covering numbers for low-rank tensors
• Theorem 4: TIHT for Low Rank Recovery rauhut2017low
• Definition 2.4: Sub-Gaussian Distribution
• Theorem 5: TensorRIP for low HOSVD-rank tensors rauhut2017low