Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Multigrid Preconditioner for Tensor Product Spline Smoothing

Martin Siebenborn, Julian Wagner

TL;DR

This paper addresses smoothing splines in multiple dimensions using tensor-product B-splines, which suffer from curse of dimensionality. It introduces a memory-efficient, matrix-free geometric multigrid preconditioned CG (MGCG) to solve the regularized least squares system $(\Phi^T\Phi + \lambda \Lambda)\alpha = \Phi^T y$. Key contributions include a matrix-free Gramian computations using Kronecker and Khatri-Rao structures, a grid-transfer framework with exact coarse-grid solves, and a V-cycle preconditioner that yields grid-independent convergence for fixed dimension. The numerical results demonstrate scalability to moderate high dimensions with scattered data and significant reductions in iterations and memory compared to standard CG.

Abstract

Uni- and bivariate data smoothing with spline functions is a well established method in nonparametric regression analysis. The extension to multivariate data is straightforward, but suffers from exponentially increasing memory and computational complexity. Therefore, we consider a matrix-free implementation of a geometric multigrid preconditioned conjugate gradient method for the regularized least squares problem resulting from tensor product B-spline smoothing with multivariate and scattered data. The algorithm requires a moderate amount of memory and is therefore applicable also for high-dimensional data. Moreover, for arbitrary but fixed dimension, we achieve grid independent convergence which is fundamental to achieve algorithmic scalability.

A Multigrid Preconditioner for Tensor Product Spline Smoothing

TL;DR

This paper addresses smoothing splines in multiple dimensions using tensor-product B-splines, which suffer from curse of dimensionality. It introduces a memory-efficient, matrix-free geometric multigrid preconditioned CG (MGCG) to solve the regularized least squares system . Key contributions include a matrix-free Gramian computations using Kronecker and Khatri-Rao structures, a grid-transfer framework with exact coarse-grid solves, and a V-cycle preconditioner that yields grid-independent convergence for fixed dimension. The numerical results demonstrate scalability to moderate high dimensions with scattered data and significant reductions in iterations and memory compared to standard CG.

Abstract

Uni- and bivariate data smoothing with spline functions is a well established method in nonparametric regression analysis. The extension to multivariate data is straightforward, but suffers from exponentially increasing memory and computational complexity. Therefore, we consider a matrix-free implementation of a geometric multigrid preconditioned conjugate gradient method for the regularized least squares problem resulting from tensor product B-spline smoothing with multivariate and scattered data. The algorithm requires a moderate amount of memory and is therefore applicable also for high-dimensional data. Moreover, for arbitrary but fixed dimension, we achieve grid independent convergence which is fundamental to achieve algorithmic scalability.

Paper Structure

This paper contains 19 sections, 43 equations, 4 figures, 3 tables, 8 algorithms.

Table of Contents

  1. Introduction
  2. Spline Smoothing in Multiple Dimensions
  3. Tensor Product Splines
  4. Tensor Product Smoothing Spline
  5. Curse of Dimensionality
  6. Tensor Product Properties
  7. Geometric Multigrid Preconditioner
  8. Memory Efficient Matrix Operations
  9. Kronecker Matrices
  10. Khatri-Rao Matrices
  11. MGCG Algorithm for Spline Smoothing
  12. Hierarchy
  13. Smoothing Iteration
  14. Grid Transfer
  15. Coarse Grid Solver
  16. ...and 4 more sections

Figures (4)

  • Figure 1: Plot of the (undisturbed) sigmoid test function for $P=1$ and $P=2$.
  • Figure 2: Number of preconditioned and unpreconditioned conjugate gradient iterations under uniform grid refinements in $P=2$ dimensions.
  • Figure 3: Eigenvalues of the (un)preconditioned coefficient matrices for $G=5$ grids and $P=2$ dimensions.
  • Figure 4: Smoothing spline approximation (left) and related residuals of the approximation (right) for $G=5$ grids and $P=2$ dimensions.