Bivariate Matrix-valued Linear Regression (BMLR): Finite-sample performance under Identifiability and Sparsity Assumptions

TL;DR

This work introduces BMLR, a bilinear matrix-valued regression framework for matrix-valued responses and predictors, and imposes identifiability through nonnegative, row-normalized $A^*$ while preserving the bilinear structure. It delivers optimization-free estimators with non-asymptotic convergence guarantees, analyzes both population (noiseless) and sample (noisy) regimes under Gaussian noise, and extends to sparse settings with hard-thresholding capable of exact support recovery. Theoretical results reveal distinct dimensionality effects: a blessing of dimensionality for $\hat{B}$ with larger row counts and a trade-off for $\hat{A}$, plus robustness of the sparse estimators to noise. Numerical simulations and CIFAR-10 denoising experiments validate the finite-sample rates and practical utility, highlighting efficient, interpretable estimation in high-dimensional matrix-structured regression tasks.

Abstract

This study explores the estimation of parameters in a matrix-valued linear regression model, where the $T$ responses $(Y_t)_{t=1}^T \in \mathbb{R}^{n \times p}$ and predictors $(X_t)_{t=1}^T \in \mathbb{R}^{m \times q}$ satisfy the relationship $Y_t = A^* X_t B^* + E_t$ for all $t = 1, \ldots, T$. In this model, $A^* \in \mathbb{R}_+^{n \times m}$ has $L_1$-normalized rows, $B^* \in \mathbb{R}^{q \times p}$, and $(E_t)_{t=1}^T$ are independent noise matrices following a matrix Gaussian distribution. The primary objective is to estimate the unknown parameters $A^*$ and $B^*$ efficiently. We propose explicit optimization-free estimators and establish non-asymptotic convergence rates to quantify their performance. Additionally, we extend our analysis to scenarios where $A^*$ and $B^*$ exhibit sparse structures. To support our theoretical findings, we conduct numerical simulations that confirm the behavior of the estimators, particularly with respect to the impact of the dimensions $n, m, p, q$, and the sample size $T$ on finite-sample performances. We complete the simulations by investigating the denoising performances of our estimators on noisy real-world images.

Figures (6)

• Figure 1: Evolution (EV) of the Frobenius norm (resp. operator norm) of $\hat{A} - A^*$ (in blue, resp. in green) and of $\hat{B} - B^*$ (in orange, resp. in red) with respect to (w.r.t.) different parameters.
• Figure 2: Original, noisy, and corrected versions of the $11^{\text{th}}$ image from the test set for $\epsilon = 0.02$.
• Figure 3: $\text{D}_{\text{o,n}}$ (blue) and $\text{D}_{\text{o,c}}$ (orange), averaged on the test set, as functions of $\epsilon$. Error bars indicate standard deviations.
• Figure 4: Empirical reconstruction errors of $\hat{A}$ and $\hat{B}$ versus model dimensions and sample size $T$, displayed on a logarithmic scale. Each curve corresponds to a given norm (Frobenius, operator, or max).
• Figure 5: Reconstruction error versus effective SNR (Frobenius norm). Lower distances indicate improved correction quality. The x-axis is expressed in dB following the definition $\mathrm{SNR}_{\mathrm{F}} = 20\log_{10}(\|X\|_{\mathrm{F}} / \|X - X_{\text{noisy}}\|_{\mathrm{F}})$.

Theorems & Definitions (31)

• Remark 2.1
• Proposition 2.2
• Remark 2.3
• Corollary 2.4
• Remark 2.5
• Lemma 3.1
• Theorem 3.3
• Lemma 3.5
• Theorem 3.6
• Theorem 3.7