Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Basis Function Selection for Computationally Efficient Predictions

Anton Kullberg, Frida Viset, Isaac Skog, Gustaf Hendeby

TL;DR

The paper addresses the high computational cost of basis-function expansions when predictive variance is computed. It introduces a prediction-time, test-input–adaptive reduction that retains only the most relevant bases within a subdomain $\Omega$, without needing extra data. The method provides deterministic and probabilistic (Gaussian) selection strategies and extends to Hilbert-space GP sparsifications using a diagonal approximation to maintain low cost. Numerical experiments demonstrate substantial speedups (e.g., $50$–$75\%$) with only modest losses in accuracy, highlighting the approach as a general, data-free form of model compression applicable to large-scale expansion models and distributed sensing tasks like multi-agent magnetic-field localization.

Abstract

Basis Function (BF) expansions are a cornerstone of any engineer's toolbox for computational function approximation which shares connections with both neural networks and Gaussian processes. Even though BF expansions are an intuitive and straightforward model to use, they suffer from quadratic computational complexity in the number of BFs if the predictive variance is to be computed. We develop a method to automatically select the most important BFs for prediction in a sub-domain of the model domain. This significantly reduces the computational complexity of computing predictions while maintaining predictive accuracy. The proposed method is demonstrated using two numerical examples, where reductions up to 50-75% are possible without significantly reducing the predictive accuracy.

Adaptive Basis Function Selection for Computationally Efficient Predictions

TL;DR

The paper addresses the high computational cost of basis-function expansions when predictive variance is computed. It introduces a prediction-time, test-input–adaptive reduction that retains only the most relevant bases within a subdomain , without needing extra data. The method provides deterministic and probabilistic (Gaussian) selection strategies and extends to Hilbert-space GP sparsifications using a diagonal approximation to maintain low cost. Numerical experiments demonstrate substantial speedups (e.g., –) with only modest losses in accuracy, highlighting the approach as a general, data-free form of model compression applicable to large-scale expansion models and distributed sensing tasks like multi-agent magnetic-field localization.

Abstract

Basis Function (BF) expansions are a cornerstone of any engineer's toolbox for computational function approximation which shares connections with both neural networks and Gaussian processes. Even though BF expansions are an intuitive and straightforward model to use, they suffer from quadratic computational complexity in the number of BFs if the predictive variance is to be computed. We develop a method to automatically select the most important BFs for prediction in a sub-domain of the model domain. This significantly reduces the computational complexity of computing predictions while maintaining predictive accuracy. The proposed method is demonstrated using two numerical examples, where reductions up to 50-75% are possible without significantly reducing the predictive accuracy.
Paper Structure (13 sections, 17 equations, 2 figures)

This paper contains 13 sections, 17 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Basis Function Selection
  3. Adaptive basis function selection
  4. Finding $\mathcal{J}$
  5. Probabilistic solution
  6. Discussion
  7. Hilbert--space GP dual selection
  8. approximation
  9. Dual parametrization
  10. Numerical Examples
  11. Radial basis functions
  12. Random function
  13. Conclusions

Figures (2)

  • Figure 1: Posterior predictive distributions in an model. The full model is given by $p(f^*)$. The posteriors $q_I(f^*)$ and $q_S(f^*)$ use reduced models with bases selected by \ref{['eq:Lbound', 'eq:simplifiedbound']}, respectively. The for the original model are depicted at the bottom of the plot and the chosen for the reduced models are depicted at the top of the plot. Note that the proportions of the are exaggerated. The subdomain $\Omega$ of interest is highlighted in gray.
  • Figure 2: Performance of the reduced model relative to the full (original) model. $\rho$ is the fraction of chosen for prediction, i.e.., $\rho=0.1$ chooses the $10\%$ most important . $L$ is the total number of basis functions available for the full model. The metrics are all relative, where blue is better and red is worse.