Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Approximate Matrix Multiplication over Sliding Window

Haoming Xian, Qintian Guo, Jun Zhang, Sibo Wang

TL;DR

This work tackles the problem of exact matrix multiplication in streaming contexts where only the most recent data should influence the result, formalizing AMM over sliding windows. The authors introduce SO-COD, a space-efficient solution that combines Co-Occurring Directions sketches with a lambda-snapshot mechanism to register only dominant co-occurring directions within the current window, achieving optimal space bounds in the normalized setting and near-optimal bounds in the unnormalized setting. They further extend the approach to unnormalized data using a multi-layer ML-SO-COD construction, achieving a space complexity of $O\left(\dfrac{d_x+d_y}{\\epsilon} \log R\right)$ and providing substantial theoretical guarantees, including a matching space lower bound. Extensive experiments on synthetic and real datasets demonstrate that SO-COD attains superior space-accuracy trade-offs compared with baselines, validating its practicality for large-scale, dynamic streaming matrix multiplication.

Abstract

Matrix multiplication is a core operation in numerous applications, yet its exact computation becomes prohibitively expensive as data scales, especially in streaming environments where timeliness is critical. In many real-world scenarios, data arrives continuously, making it essential to focus on recent information via sliding windows. While existing approaches offer approximate solutions, they often suffer from suboptimal space complexities when extended to the sliding-window setting. In this work, we introduce SO-COD, a novel algorithm for approximate matrix multiplication (AMM) in the sliding-window streaming setting, where only the most recent data is retained for computation. Inspired by frequency estimation over sliding windows, our method tracks significant contributions, referred to as snapshots, from incoming data and efficiently updates them as the window advances. Given matrices \(\boldsymbol{X} \in \mathbb{R}^{d_x \times n}\) and \(\boldsymbol{Y} \in \mathbb{R}^{d_y \times n}\) for computing \(\boldsymbol{X} \boldsymbol{Y}^T\), we analyze two data settings. In the \emph{normalized} setting, where each column of the input matrices has a unit \(L_2\) norm, SO-COD achieves an optimal space complexity of \( O\left(\frac{d_x+d_y}ε\right) \). In the \emph{unnormalized} setting, where the square of column norms vary within a bounded range \([1, R]\), we show that the space requirement is \( O\left(\frac{d_x+d_y}ε\log R\right) \), which matches the theoretical lower bound for an \(ε\)-approximation guarantee. Extensive experiments on synthetic and real-world datasets demonstrate that SO-COD effectively balances space cost and approximation error, making it a promising solution for large-scale, dynamic streaming matrix multiplication.

Optimal Approximate Matrix Multiplication over Sliding Window

TL;DR

This work tackles the problem of exact matrix multiplication in streaming contexts where only the most recent data should influence the result, formalizing AMM over sliding windows. The authors introduce SO-COD, a space-efficient solution that combines Co-Occurring Directions sketches with a lambda-snapshot mechanism to register only dominant co-occurring directions within the current window, achieving optimal space bounds in the normalized setting and near-optimal bounds in the unnormalized setting. They further extend the approach to unnormalized data using a multi-layer ML-SO-COD construction, achieving a space complexity of and providing substantial theoretical guarantees, including a matching space lower bound. Extensive experiments on synthetic and real datasets demonstrate that SO-COD attains superior space-accuracy trade-offs compared with baselines, validating its practicality for large-scale, dynamic streaming matrix multiplication.

Abstract

Matrix multiplication is a core operation in numerous applications, yet its exact computation becomes prohibitively expensive as data scales, especially in streaming environments where timeliness is critical. In many real-world scenarios, data arrives continuously, making it essential to focus on recent information via sliding windows. While existing approaches offer approximate solutions, they often suffer from suboptimal space complexities when extended to the sliding-window setting. In this work, we introduce SO-COD, a novel algorithm for approximate matrix multiplication (AMM) in the sliding-window streaming setting, where only the most recent data is retained for computation. Inspired by frequency estimation over sliding windows, our method tracks significant contributions, referred to as snapshots, from incoming data and efficiently updates them as the window advances. Given matrices and for computing , we analyze two data settings. In the \emph{normalized} setting, where each column of the input matrices has a unit norm, SO-COD achieves an optimal space complexity of \( O\left(\frac{d_x+d_y}ε\right) \). In the \emph{unnormalized} setting, where the square of column norms vary within a bounded range , we show that the space requirement is \( O\left(\frac{d_x+d_y}ε\log R\right) \), which matches the theoretical lower bound for an -approximation guarantee. Extensive experiments on synthetic and real-world datasets demonstrate that SO-COD effectively balances space cost and approximation error, making it a promising solution for large-scale, dynamic streaming matrix multiplication.

Paper Structure

This paper contains 23 sections, 7 theorems, 17 equations, 3 figures, 1 table, 9 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notations and Basic Concepts
  4. Problem Definition
  5. Co-Occurring Directions
  6. $\boldsymbol{\lambda}$-Snapshot Method
  7. Our Solution
  8. Algorithm Description
  9. A Fast Update Algorithm
  10. General Unnormalized Model
  11. Algorithm Description
  12. Space Lower Bound
  13. Experiments
  14. Baseline Algorithms
  15. Experiments Setup
  16. ...and 8 more sections

Key Result

Theorem 1

The output of Co-occurring Directions (Algorithm alg:cod) provides correlation sketch matrices $(\boldsymbol{B}_X \in \mathbb{R}^{d_x \times l}, \boldsymbol{B}_Y \in \mathbb{R}^{d_y \times l})$ for $(\boldsymbol{X} \in \mathbb{R}^{d_x \times n}, \boldsymbol{Y} \in \mathbb{R}^{d_y \times n})$, where Algorithm alg:cod runs in $O(n(d_x + d_y)l)$ time using $O((d_x + d_y)l)$ space.

Figures (3)

  • Figure 1: Maximum Sketch Size vs. Maximum Error.
  • Figure 2: Maximum Sketch Size vs. Average Error.
  • Figure 3: $\log_{10}(1/\epsilon)$ vs. Maximum Sketch Size ($\log_{10}(\frac{1}{0.25}) \approx 0.6$, and $\log_{10}(\frac{1}{0.016}) \approx 1.8$).

Theorems & Definitions (9)

  • Definition 1: AMM over a Sliding Window
  • Theorem 1
  • Lemma 1
  • Theorem 2
  • Theorem 3
  • Lemma 2: Lower Bound for AMM LuoCXY21
  • Lemma 3: LuoCXY21
  • Theorem 4: Lower Bound for AMM over Sliding Window
  • Example 1