Broadcast Product: Shape-aligned Element-wise Multiplication and Beyond

TL;DR

The paper formalizes a broadcast product operator $\boxdot$ that extends the Hadamard product to shape-aligned tensor pairs by duplicating elements to match shapes, enabling precise mathematical representations of numpy-style broadcasting. It establishes the broadcast condition, defines duplication via $\boldsymbol{\mathcal X}^{\square}$, and derives key algebraic properties and norm relations. Building on this, it introduces the broadcast decomposition (BD) and shows how ALS/HALS can optimize BD objectives, with a synthetic study suggesting BD can achieve accurate low-parametric approximations where traditional tensor decompositions struggle. The work offers a principled, notation-clean framework for concise expression and optimization of broadcasted tensor operations, with potential applications in dimensionality reduction and beyond.

Abstract

We propose a new operator defined between two tensors, the broadcast product. The broadcast product calculates the Hadamard product after duplicating elements to align the shapes of the two tensors. Complex tensor operations in libraries like \texttt{numpy} can be succinctly represented as mathematical expressions using the broadcast product. Finally, we propose a novel tensor decomposition using the broadcast product, highlighting its potential applications in dimensionality reduction.

Figures (3)

• Figure 1: Example of the broadcast product of a third-order tensor and a matrix
• Figure 2: Dimensionality reduction of a synthetic tensor: This suggests the existence of a tensor structure that favors broadcast decomposition.
• Figure 3: The broadcast product of a third-order tensor and a matrix: $\bm{\mathcal{X}} \boxdot \bm{Y} = \bm{\mathcal{X}} \odot \mathrm{fold}_1 (\bm{1}_K^\top \otimes \bm{Y}) = \mathrm{fold}_3(\bm X_{(3)} \mathrm{diag}(\mathrm{vec}(\bm Y)))$