Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hadamard-Hitchcock decompositions: identifiability and computation

Alessandro Oneto, Nick Vannieuwenhoven

TL;DR

This work introduces Hadamard–Hitchcock decompositions (HHDs), which express tensors as the elementwise product of Hitchcock decompositions, and links them to discrete RBMs. It establishes generic identifiability for low‑rank HHDs by leveraging a reshaped Kruskal criterion and introduces rank‑1 permutations as a tool to recover the individual rank‑1 terms within Hadamard factors. A practical, non‑optimization‑based algorithm is developed to compute the unique HHD from a given tensor by first extracting a rank‑R CPD, then determining a rank‑1 permutation and reconstructing the Hadamard factors via a Hadamard–Hitchcock tensor (HHT). Extensive numerical experiments in Julia demonstrate accurate reconstruction, discuss runtime trade‑offs, and illustrate tolerance to mild model violations, including a synthetic RBM example. The results advance identifiability theory for HHDs and provide a concrete, implementable pipeline for decomposing tensors into Hadamard products of CPDs with potential applications to probabilistic graphical models and beyond.

Abstract

A Hadamard-Hitchcock decomposition of a multidimensional array is a decomposition that expresses the latter as a Hadamard product of several tensor rank decompositions. Such decompositions can encode probability distributions that arise from statistical graphical models associated to complete bipartite graphs with one layer of observed random variables and one layer of hidden ones, usually called restricted Boltzmann machines. We establish generic identifiability of Hadamard-Hitchcock decompositions by exploiting the reshaped Kruskal criterion for tensor rank decompositions. A flexible algorithm leveraging existing decomposition algorithms for tensor rank decomposition is introduced for computing a Hadamard-Hitchcock decomposition. Numerical experiments illustrate its computational performance and numerical accuracy.

Hadamard-Hitchcock decompositions: identifiability and computation

TL;DR

This work introduces Hadamard–Hitchcock decompositions (HHDs), which express tensors as the elementwise product of Hitchcock decompositions, and links them to discrete RBMs. It establishes generic identifiability for low‑rank HHDs by leveraging a reshaped Kruskal criterion and introduces rank‑1 permutations as a tool to recover the individual rank‑1 terms within Hadamard factors. A practical, non‑optimization‑based algorithm is developed to compute the unique HHD from a given tensor by first extracting a rank‑R CPD, then determining a rank‑1 permutation and reconstructing the Hadamard factors via a Hadamard–Hitchcock tensor (HHT). Extensive numerical experiments in Julia demonstrate accurate reconstruction, discuss runtime trade‑offs, and illustrate tolerance to mild model violations, including a synthetic RBM example. The results advance identifiability theory for HHDs and provide a concrete, implementable pipeline for decomposing tensors into Hadamard products of CPDs with potential applications to probabilistic graphical models and beyond.

Abstract

A Hadamard-Hitchcock decomposition of a multidimensional array is a decomposition that expresses the latter as a Hadamard product of several tensor rank decompositions. Such decompositions can encode probability distributions that arise from statistical graphical models associated to complete bipartite graphs with one layer of observed random variables and one layer of hidden ones, usually called restricted Boltzmann machines. We establish generic identifiability of Hadamard-Hitchcock decompositions by exploiting the reshaped Kruskal criterion for tensor rank decompositions. A flexible algorithm leveraging existing decomposition algorithms for tensor rank decomposition is introduced for computing a Hadamard-Hitchcock decomposition. Numerical experiments illustrate its computational performance and numerical accuracy.
Paper Structure (23 sections, 16 theorems, 84 equations, 7 figures, 1 table, 6 algorithms)

This paper contains 23 sections, 16 theorems, 84 equations, 7 figures, 1 table, 6 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. CPDs
  4. HHDs
  5. From HHD to CPD and back
  6. Rank-1 permutations
  7. Preservers and permutations
  8. Computing a rank-1 permutation
  9. Identifiability of low-rank HHDs
  10. A numerical algorithm for computing low-rank HHDs
  11. Basic version
  12. Advanced version
  13. Time complexity
  14. Numerical experiments
  15. Implementation details
  16. ...and 8 more sections

Key Result

Theorem 1

Let $\mathbf{r}\in\mathbb{N}^m$ and $R=r_1\cdots r_m$. Let $\mathpzc{T}$ admit a generic eqn_hhd. If $\mathpzc{T}$ is $R$-identifiable, then the rank-$1$ tensors $\mathpzc{A}_{k i_k}$ in the eqn_hhd are uniquely determined up to the following symmetries: Moreover, if $R$ falls within the range of the reshaped Kruskal criterion COV17, then the generic $\mathpzc{T}$ admitting a decomposition as in

Figures (7)

  • Figure 1: The computational time and estimated asymptotic time complexity for computing rank-$1$ permutations of tensors containing $R$ elements as explained in \ref{['sec_sub_rk1permexp']}.
  • Figure 2: Relative backward error of the CPD and HHD for random decomposition problems into $(r_1,r_2)$-HHDs from \ref{['sec_sub_hhdexper']}.
  • Figure 3: The execution time (s) of \ref{['alg_hhd']} spent in the CPD phase (line \ref{['line_cpd']}) and the HHD phase (all other lines) for random decomposition problems into rank-$(r_1,r_2)$ HHDs from \ref{['sec_sub_hhdexper']}.
  • Figure 4: Relative backward error of the CPD and HHD for random decomposition problems into $(r_1,r_2)$-HHDs from \ref{['sec_sub_comparisonpba']}, where the CPD is computed with two different algorithms. The scale is the same in all plots.
  • Figure 5: The execution time (s) of \ref{['alg_hhd']} spent in the CPD phase (line \ref{['line_cpd']}) and the HHD phase (all other lines) for random decomposition problems into rank-$(r_1,r_2)$ HHDs from \ref{['sec_sub_comparisonpba']}.
  • ...and 2 more figures

Theorems & Definitions (37)

  • Definition 1.1
  • Remark 1.2
  • Theorem : Identifiability of generic \ref{['eqn_hhd']}s
  • Definition 2.1
  • Lemma 2.2: COV17
  • Definition 3.1
  • Lemma 3.2
  • proof
  • Definition 3.3
  • Proposition 3.4
  • ...and 27 more