Tensor BM-Decomposition for Compression and Analysis of Video Data

TL;DR

This work develops a tensor Bhattacharya-Mesner decomposition (BMD) framework to compress and analyze video data by expressing a video tensor as a sum of BM-rank-1 terms via the BM-product. It establishes theoretical connections to rank-revealing matrix factorizations, introduces a generative low BM-rank video model, and delivers a regularized ALS algorithm (BMD-ALS) with scalable, parallelizable updates that effectively separate background from foreground. The paper further extends the approach to fourth-order, color video through a BMP4-like construction and corresponding regularized ALS, enabling coherent color-channel compression and motion extraction. Extensive numerical results on grayscale and color videos show superior compression and cleaner background/foreground separation compared to matrix-based DMD and tensor SS-SVD baselines. Overall, the method provides a principled, scalable framework for spatiotemporal video compression and analysis with potential applicability to broader high-dimensional data.

Abstract

Given tensors $\boldsymbol{\mathscr{A}}, \boldsymbol{\mathscr{B}}, \boldsymbol{\mathscr{C}}$ of size $m \times 1 \times n$, $m \times p \times 1$, and $1\times p \times n$, respectively, their Bhattacharya-Mesner (BM) product will result in a third-order tensor of dimension $m \times p \times n$ and BM-rank of 1 (Mesner and Bhattacharya, 1990). Thus, if an arbitrary $m \times p \times n$ third-order tensor can be written as a sum of a small number, relative to $m,p,n$, of such BM-rank 1 terms, this BM-decomposition (BMD) offers an implicitly compressed representation of the tensor. In this paper, we first show that grayscale surveillance video can be accurately captured by a low BM-rank decomposition and give methods for efficiently computing this decomposition. To this end, we first give results that connect rank-revealing matrix factorizations to the BMD. Next, we present a generative model that illustrates that spatio-temporal video data can be expected to have low BM-rank. We combine these observations to derive a regularized alternating least squares (ALS) algorithm to compute an approximate BMD of the video tensor. The algorithm itself is highly parallelizable since the bulk of the computations break down into relatively small regularized least squares problems that can be solved independently. Extensive numerical results compared against the state-of-the-art matrix-based DMD for surveillance video separation show our algorithms can consistently produce results with superior compression properties while simultaneously providing better separation of stationary and non-stationary features in the data. We then introduce a new type of BM-product suitable for color video and provide an algorithm that shows an impressive ability to extract important temporal information from color video while simultaneously compressing the data.

Figures (12)

• Figure 1: Slices and tube fibers of a third-order tensor of size $m \times p \times n$ and corresponding indexing in Matlab notation.
• Figure 1: Illustration of the generative spatiotemporal video model with two constant valued, same intensity, rectangular objects moving in the same horizontal direction, on a constant background.
• Figure 1: Video frames selected to display from the six testing videos.
• Figure 1: Color video background/foreground separation by the fourth-order $\text{BMD-ALS}_{SVD}$ method.
• Figure 2: (a) Illustration of the tensor ${\hbox{\tt Tvec}}(\boldsymbol{\mathscr{X}})$ and ${\hbox{\tt Tfold}}({\bm{\mathbf{x}}})$ operations. The vectorization and the reshaping of the first three tube fibers $\boldsymbol{\mathscr{X}}_{1,1,:}, \boldsymbol{\mathscr{X}}_{1,2,:},$ and $\boldsymbol{\mathscr{X}}_{1,3,:}$ are shown in this figure, and the rest are omitted. (b) Illustration of the tensor ${\hbox{\tt Mat}}$ operation that flattens the two tensors $\boldsymbol{\mathscr{A}}\in \mathbb{R}^{m\times \ell \times n},\boldsymbol{\mathscr{B}}\in \mathbb{R}^{\ell \times p \times n}$ into a block-diagonal matrix ${\bm{\mathbf{H}}}\in \mathbb{R}^{mpn \times mp\ell}$.

Theorems & Definitions (28)

• Definition 2.1
• Remark 1
• Definition 2.2: BM-rank gnang2020bhattacharya
• Definition 2.3
• Theorem 3.1
• Proof 1
• Corollary 3.2
• Theorem 3.3
• Corollary 3.4
• Theorem 4.1