Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Tensor Inference

Xin Wen, Will Wei Sun, Yichen Zhang

TL;DR

This work develops an online framework for real-time inference on low-rank tensor data, combining a Tucker-structured online SGD estimator with an online debiasing procedure to produce valid confidence intervals without data storage. The method achieves near-minimax online estimation rates and establishes asymptotic normality for general tensor linear forms, enabling sequential hypothesis testing in streaming settings. The authors provide non-asymptotic convergence guarantees, a two-stage step-size schedule, and online variance estimation, plus practical demonstrations on simulated data and large-scale Taobao user–item interactions. The approach yields scalable, memory-efficient Tensor regression with principled uncertainty quantification, facilitating data-driven, on-the-fly decision making in dynamic environments.

Abstract

Contemporary applications, such as recommendation systems and mobile health monitoring, require real-time processing and analysis of sequentially arriving high-dimensional tensor data. Traditional offline learning, involving the storage and utilization of all data in each computational iteration, becomes impractical for these tasks. Furthermore, existing low-rank tensor methods lack the capability for online statistical inference, which is essential for real-time predictions and informed decision-making. This paper addresses these challenges by introducing a novel online inference framework for low-rank tensors. Our approach employs Stochastic Gradient Descent (SGD) to enable efficient real-time data processing without extensive memory requirements. We establish a non-asymptotic convergence result for the online low-rank SGD estimator, nearly matches the minimax optimal estimation error rate of offline models. Furthermore, we propose a simple yet powerful online debiasing approach for sequential statistical inference. The entire online procedure, covering both estimation and inference, eliminates the need for data splitting or storing historical data, making it suitable for on-the-fly hypothesis testing. In our analysis, we control the sum of constructed super-martingales to ensure estimates along the entire solution path remain within the benign region. Additionally, a novel spectral representation tool is employed to address statistical dependencies among iterative estimates, establishing the desired asymptotic normality.

Online Tensor Inference

TL;DR

This work develops an online framework for real-time inference on low-rank tensor data, combining a Tucker-structured online SGD estimator with an online debiasing procedure to produce valid confidence intervals without data storage. The method achieves near-minimax online estimation rates and establishes asymptotic normality for general tensor linear forms, enabling sequential hypothesis testing in streaming settings. The authors provide non-asymptotic convergence guarantees, a two-stage step-size schedule, and online variance estimation, plus practical demonstrations on simulated data and large-scale Taobao user–item interactions. The approach yields scalable, memory-efficient Tensor regression with principled uncertainty quantification, facilitating data-driven, on-the-fly decision making in dynamic environments.

Abstract

Contemporary applications, such as recommendation systems and mobile health monitoring, require real-time processing and analysis of sequentially arriving high-dimensional tensor data. Traditional offline learning, involving the storage and utilization of all data in each computational iteration, becomes impractical for these tasks. Furthermore, existing low-rank tensor methods lack the capability for online statistical inference, which is essential for real-time predictions and informed decision-making. This paper addresses these challenges by introducing a novel online inference framework for low-rank tensors. Our approach employs Stochastic Gradient Descent (SGD) to enable efficient real-time data processing without extensive memory requirements. We establish a non-asymptotic convergence result for the online low-rank SGD estimator, nearly matches the minimax optimal estimation error rate of offline models. Furthermore, we propose a simple yet powerful online debiasing approach for sequential statistical inference. The entire online procedure, covering both estimation and inference, eliminates the need for data splitting or storing historical data, making it suitable for on-the-fly hypothesis testing. In our analysis, we control the sum of constructed super-martingales to ensure estimates along the entire solution path remain within the benign region. Additionally, a novel spectral representation tool is employed to address statistical dependencies among iterative estimates, establishing the desired asymptotic normality.
Paper Structure (60 sections, 26 theorems, 329 equations, 7 figures, 6 algorithms)

This paper contains 60 sections, 26 theorems, 329 equations, 7 figures, 6 algorithms.

Table of Contents

  1. Introduction
  2. Related Literature
  3. Online Low-Rank Tensor SGD
  4. Low-rank Tensor Models
  5. Online Low-Rank Tensor Estimation
  6. Convergence Analysis
  7. Online Statistical Inference for Low-Rank Tensors
  8. Constructing De-biased Estimators
  9. Asymptotic Normality of $\hat{h}^{(n)}$
  10. Online Parameter Inference of $\hat{h}^{(n)}$
  11. Numerical Simulations
  12. Real Data Analysis
  13. Additional Related Literature
  14. Extended Simulation Results
  15. Extension to General-order Tensor Case
  16. ...and 45 more sections

Key Result

Theorem 3.1

For any constant $\alpha \in (0, 1)$, we define the learning rate $\eta_t = \eta_0 \left( \max \left\{t, t^{\star}\right\} \right)^{-\alpha}$ for some constant $\eta_0$, where $t^{\star} = \left(C_{\max} \mathsf{df}\right)^{1/\alpha}$. The tensor $\mathcal{T}^{(t)}= \mathcal{G}^{(t)} \times_{k\in[3] where $\mathsf{df} := r_1 r_2 r_3 + \sum_{k=1}^3 p_k r_k$, and $C_1$ is positive constant.

Figures (7)

  • Figure 1: Error analysis for our online tensor estimation across different dimensions, ranks, and noise levels.
  • Figure 2: Error analysis for our online tensor estimation across different dimensions $p$ and ranks $r$.
  • Figure 3: Coverage probabilities for hypothesis tests in \ref{['equ:example-1']} (left plot) and \ref{['equ:example-2']} (right plot).
  • Figure 4: Histogram of normal approximation over 1000 independent trails for different ranks.
  • Figure 5: Histogram of normal approximation over 1000 independent trails for different noise levels.
  • ...and 2 more figures

Theorems & Definitions (32)

  • Theorem 3.1
  • Remark 3.1: Two-Stage Step-Size Schedule
  • Theorem 4.1
  • Theorem 4.2
  • Lemma E.1
  • Lemma E.2
  • Lemma E.3
  • Lemma E.4
  • Lemma E.5
  • Lemma E.6
  • ...and 22 more