Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tucker Diffusion Model for High-dimensional Tensor Generation

Jianhua Guo, Xinbing Kong, Zeyu Li, Junfan Mao

Abstract

Statistical inference on large-dimensional tensor data has been extensively studied in the literature and widely used in economics, biology, machine learning, and other fields, but how to generate a structured tensor with a target distribution is still a new problem. As profound AI generators, diffusion models have achieved remarkable success in learning complex distributions. However, their extension to generating multi-linear tensor-valued observations remains underexplored. In this work, we propose a novel Tucker diffusion model for learning high-dimensional tensor distributions. We show that the score function admits a structured decomposition under the low Tucker rank assumption, allowing it to be both accurately approximated and efficiently estimated using a carefully tailored tensor-shaped architecture named Tucker-Unet. Furthermore, the distribution of generated tensors, induced by the estimated score function, converges to the true data distribution at a rate depending on the maximum of tensor mode dimensions, thereby offering a clear theoretical advantage over the naive vectorized approach, which has a product dependence. Empirically, compared to existing approaches, the Tucker diffusion model demonstrates strong practical potential in synthetic and real-world tensor generation tasks, achieving comparable and sometimes even superior statistical performance with significantly reduced training and sampling costs.

Tucker Diffusion Model for High-dimensional Tensor Generation

Abstract

Statistical inference on large-dimensional tensor data has been extensively studied in the literature and widely used in economics, biology, machine learning, and other fields, but how to generate a structured tensor with a target distribution is still a new problem. As profound AI generators, diffusion models have achieved remarkable success in learning complex distributions. However, their extension to generating multi-linear tensor-valued observations remains underexplored. In this work, we propose a novel Tucker diffusion model for learning high-dimensional tensor distributions. We show that the score function admits a structured decomposition under the low Tucker rank assumption, allowing it to be both accurately approximated and efficiently estimated using a carefully tailored tensor-shaped architecture named Tucker-Unet. Furthermore, the distribution of generated tensors, induced by the estimated score function, converges to the true data distribution at a rate depending on the maximum of tensor mode dimensions, thereby offering a clear theoretical advantage over the naive vectorized approach, which has a product dependence. Empirically, compared to existing approaches, the Tucker diffusion model demonstrates strong practical potential in synthetic and real-world tensor generation tasks, achieving comparable and sometimes even superior statistical performance with significantly reduced training and sampling costs.

Paper Structure

This paper contains 18 sections, 4 theorems, 36 equations, 4 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Initial tensor data distribution
  3. Contributions
  4. Organization and notations
  5. Tucker Diffusion Model
  6. Score matching
  7. Score decomposition and Tucker-Unet
  8. Backward sampling
  9. Theory
  10. Theory of Score Approximation
  11. Theory of Score Estimation
  12. Theory of Distribution Estimation
  13. Numerical Simulation
  14. Real Data Cases
  15. Molecular generation
  16. ...and 3 more sections

Key Result

Lemma 1

Under Assumption assum:1, the score function $\nabla\log p_t(\mathbf{X}_{t})$ can be decomposed into a subspace score and a complement score as where $p_{\mathrm{core}}^{t}(\mathbf{g}_{t})\coloneqq\int\phi\left(\mathbf{g}_{t}; \alpha_t\mathbf{f},\Sigma_{A_{t}}\right) p_{\mathrm{core}}(\mathbf{f}) \mathrm{d}\mathbf{f}$ and $\mathrm{Tucker}(\cdot)\coloneqq\mathrm{reshape}\left\{\cdot;\{r_{d}\}_{d\i

Figures (4)

  • Figure 1: Visualization of the homogeneous Tucker-Unet by deactivating the noise heterogeneity structure. The details of Tucker-Unet can be found in Section \ref{['sec:tuckerunet']}. The proposed network consists of a Tucker encoder and decoder, a core-Net and a shortcut skip connection towards the output.
  • Figure 2: A visual demonstration of the forward and backward processes of a diffusion model using a real-world medical image.
  • Figure 3: Real samples from the OrganAMNIST dataset and the generated samples using the low-Tucker-rank diffusion. The Tucker diffusion model is able to retrieve the high-dimensional distribution of these medical images with significantly smaller training and sampling costs.
  • Figure 4: Pickup (PU) and drop-off (DO) distribution patterns acquired from the generated dataset (1280 days) and the real test dataset (356 days) at different hours. The low-dimensional tensor diffusion model is able to correctly identify the airport zones and Manhattan’s major tourist, business, and hospitality districts.

Theorems & Definitions (4)

  • Lemma 1: Score decomposition lemma
  • Theorem 1: Tucker score approximation
  • Theorem 2: Tucker score estimation
  • Theorem 3: Tensor distribution estimation