Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Linear to multi-linear algebra and systems using tensors

Divyanshu Pandey, Adithya Venugopal, Harry Leib

TL;DR

The paper surveys tensor algebra based on the Einstein contracted product to extend linear algebra concepts to multi-domain signals and systems. It introduces and links tensor inverses, SVD/EVD, LU decompositions, tensor networks, and contracted convolution as a cohesive toolkit. It also develops discrete and continuous multi-linear tensor system theory, with stability criteria and transfer-function notions, and demonstrates a MIMO-CDMA example where a tensor MMSE receiver exploits cross-domain couplings. Overall, it presents a tutorial-style entry point enabling researchers to model and design multi-domain systems without resorting to vectorization while preserving mode structure.

Abstract

In past few decades, tensor algebra also known as multi-linear algebra has been developed and customized as a tool to be used for various engineering applications. In particular, with the help of a special form of tensor contracted product, known as the Einstein Product and its properties, many of the known concepts from Linear Algebra could be extended to a multi-linear setting. This enables to define the notions of multi-linear system theory where the input, output signals and the system are multi-domain in nature. This paper provides an overview of tensor algebra tools which can be seen as an extension of linear algebra, at the same time highlighting the difference and advantages that the multi-linear setting brings forth. In particular, the notion of tensor inversion, tensor singular value and tensor Eigenvalue decomposition using the Einstein product is explained. In addition, this paper also introduces the notion of contracted convolution in both discrete and continuous multi-linear system tensors. Tensor Networks representation of various tensor operations is also presented. Also, application of tensor tools in developing transceiver schemes for multi-domain communication systems, with an example of MIMO CDMA systems, is presented. Thus this paper acts as an entry point tutorial for graduate students whose research involves multi-domain or multi-modal signals and systems.

Linear to multi-linear algebra and systems using tensors

TL;DR

The paper surveys tensor algebra based on the Einstein contracted product to extend linear algebra concepts to multi-domain signals and systems. It introduces and links tensor inverses, SVD/EVD, LU decompositions, tensor networks, and contracted convolution as a cohesive toolkit. It also develops discrete and continuous multi-linear tensor system theory, with stability criteria and transfer-function notions, and demonstrates a MIMO-CDMA example where a tensor MMSE receiver exploits cross-domain couplings. Overall, it presents a tutorial-style entry point enabling researchers to model and design multi-domain systems without resorting to vectorization while preserving mode structure.

Abstract

In past few decades, tensor algebra also known as multi-linear algebra has been developed and customized as a tool to be used for various engineering applications. In particular, with the help of a special form of tensor contracted product, known as the Einstein Product and its properties, many of the known concepts from Linear Algebra could be extended to a multi-linear setting. This enables to define the notions of multi-linear system theory where the input, output signals and the system are multi-domain in nature. This paper provides an overview of tensor algebra tools which can be seen as an extension of linear algebra, at the same time highlighting the difference and advantages that the multi-linear setting brings forth. In particular, the notion of tensor inversion, tensor singular value and tensor Eigenvalue decomposition using the Einstein product is explained. In addition, this paper also introduces the notion of contracted convolution in both discrete and continuous multi-linear system tensors. Tensor Networks representation of various tensor operations is also presented. Also, application of tensor tools in developing transceiver schemes for multi-domain communication systems, with an example of MIMO CDMA systems, is presented. Thus this paper acts as an entry point tutorial for graduate students whose research involves multi-domain or multi-modal signals and systems.
Paper Structure (24 sections, 6 theorems, 60 equations, 12 figures)

This paper contains 24 sections, 6 theorems, 60 equations, 12 figures.

Table of Contents

  1. Introduction
  2. Fundamentals of tensors and notation
  3. Notations
  4. Definitions and tensor operations
  5. Tensor Products
  6. Transpose, Hermitian and Inverse of a Tensor
  7. Function Tensors
  8. Discrete Time Signal Tensors
  9. Continuous Time Signal Tensors
  10. Tensor Networks
  11. Illustration of Contraction Products
  12. Illustration of Contracted Convolutions
  13. Tensor Decompositions
  14. Tensor Singular Value Decomposition (SVD)
  15. Tensor Eigen Value Decomposition (EVD)
  16. ...and 9 more sections

Key Result

Lemma 1

For tensors $\mathscr{A} \in \mathbb{C}^{I_1 \times \dots \times I_N \times J_1 \times \dots \times J_M}$ and $\mathscr{B} \in \mathbb{C}^{J_1 \times \dots \times J_M \times K_1 \times \dots \times K_P}$ under the transformation from (Transform), the following holds:

Figures (12)

  • Figure 1: Pseudo-upper triangular Tensor
  • Figure 2: TN diagram representation of (a) vector of size $I_1$, (b) matrix of size $I_1 \times I_2$, (c) order 3 tensor of size $I_1 \times I_2 \times I_3$, (d) order $N$ tensor of size $I_1 \times \dots \times I_N$
  • Figure 3: TN representation of Contracted Product (a) mode-$n$ product between $\mathscr{A} \in \mathbb{C}^{I_1 \times \dots \times I_N}$ and $\text{U} \in \mathbb{C}^{J \times I_n}$ (b) between tensor $\mathscr{A},\mathscr{B} \in \mathbb{C}^{I \times J \times K}$ over all the three modes (inner product) (c) between $\mathscr{A} \in \mathbb{C}^{I \times J \times K \times L \times M}$ and $\mathscr{B} \in \mathbb{C}^{J \times P \times L \times N}$ as $\{\mathscr{A},\mathscr{B}\}_{\{2,4;1,3\}}$, (d) Einstein product between tensors $\mathscr{A} \in \mathbb{C}^{I_1 \times \dots \times I_P \times K_1 \times \dots \times K_N}$ and $\mathscr{B} \in \mathbb{C}^{K_1 \times \dots \times K_N \times J_1 \times \dots \times J_M}$.
  • Figure 4: TN representation of Contracted Convolution from \ref{['dfeeq3']}.
  • Figure 5: Compact TN representation of Contracted Convolution from \ref{['dfeeq3']}.
  • ...and 7 more figures

Theorems & Definitions (29)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Definition 7
  • Definition 8
  • Definition 9
  • Definition 10
  • ...and 19 more