Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sketch 'n Solve: An Efficient Python Package for Large-Scale Least Squares Using Randomized Numerical Linear Algebra

Alex Lavaee

TL;DR

Sketch 'n Solve, an open-source Python package that implements efficient randomized numerical linear algebra techniques for solving large-scale least squares problems, shows particular promise for applications in machine learning optimization, signal processing, and scientific computing.

Abstract

We present Sketch 'n Solve, an open-source Python package that implements efficient randomized numerical linear algebra (RandNLA) techniques for solving large-scale least squares problems. While sketch-and-solve algorithms have demonstrated theoretical promise, their practical adoption has been limited by the lack of robust, user-friendly implementations. Our package addresses this gap by providing an optimized implementation built on NumPy and SciPy, featuring both dense and sparse sketching operators with a clean API. Through extensive benchmarking, we demonstrate that our implementation achieves up to 50x speedup over traditional LSQR while maintaining high accuracy, even for ill-conditioned matrices. The package shows particular promise for applications in machine learning optimization, signal processing, and scientific computing.

Sketch 'n Solve: An Efficient Python Package for Large-Scale Least Squares Using Randomized Numerical Linear Algebra

TL;DR

Sketch 'n Solve, an open-source Python package that implements efficient randomized numerical linear algebra techniques for solving large-scale least squares problems, shows particular promise for applications in machine learning optimization, signal processing, and scientific computing.

Abstract

We present Sketch 'n Solve, an open-source Python package that implements efficient randomized numerical linear algebra (RandNLA) techniques for solving large-scale least squares problems. While sketch-and-solve algorithms have demonstrated theoretical promise, their practical adoption has been limited by the lack of robust, user-friendly implementations. Our package addresses this gap by providing an optimized implementation built on NumPy and SciPy, featuring both dense and sparse sketching operators with a clean API. Through extensive benchmarking, we demonstrate that our implementation achieves up to 50x speedup over traditional LSQR while maintaining high accuracy, even for ill-conditioned matrices. The package shows particular promise for applications in machine learning optimization, signal processing, and scientific computing.
Paper Structure (16 sections, 2 equations, 5 figures)

This paper contains 16 sections, 2 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Context: Randomized Numerical Linear Algebra
  3. The Sketch-and-Solve Paradigm
  4. Dense Sketching Operators
  5. Sparse Sketching Operators
  6. The Least Squares Problem
  7. Motivation and Contributions
  8. Implementation
  9. API Design
  10. Optimization Techniques
  11. Experimental Results
  12. Benchmark Setup
  13. Performance Analysis
  14. Applications
  15. Code Availability
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Dense sketch matrix
  • Figure 2: Sparse sketch matrix
  • Figure 3: Runtime comparison between SAP-SAS, SAA-SAS, and LSQR (baseline).
  • Figure 4: Forward error comparison between SAP-SAS, SAA-SAS, and LSQR (baseline).
  • Figure 5: Residual error comparison between SAP-SAS, SAA-SAS, and LSQR (baseline).