Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A class of kernel-based scalable algorithms for data science

Philippe G. LeFloch, Jean-Marc Mercier, Shohruh Miryusupov

TL;DR

The paper addresses the scalability gap of kernel-based RKHS methods in data science by introducing a multi-scale divide-and-conquer framework that preserves reproducibility and error bounds. It advances a suite of general-purpose algorithms (greedy, permutation, explicit descent) and clustering strategies (sharp discrepancy sequences and balanced clustering) to enable efficient regression, interpolation, and optimal transport on large/high-dimensional datasets. It also offers encoder–decoder sampling and conditional distribution modeling to handle distribution-to-distribution mappings and conditional generation, with a unifying OT-like framework. Numerical experiments across clustering, MNIST classification, multiscale OT, conditional sampling, and the Bachelier test demonstrate practical performance and throughputs, supported by an open-source CodPy library for reproducible research.

Abstract

We present several generative and predictive algorithms based on the RKHS (reproducing kernel Hilbert spaces) methodology, which, most importantly, are scale up efficiently with large datasets or high-dimensional data. It is well recognized that the RKHS methodology leads one to efficient and robust algorithms for numerous tasks in data science, statistics, and scientific computation. However, the implementations existing the literature are often difficult to scale up for encompassing large datasets. In this paper, we introduce a simple and robust, divide-and-conquer methodology. It applies to large scale datasets and relies on several kernel-based algorithms, which distinguish between various extrapolation, interpolation, and optimal transport steps. We argue how to select the suitable algorithm in specific applications thanks to a feedback of performance criteria. Our primary focus is on applications and problems arising in industrial contexts, such as generating meshes for efficient numerical simulations, designing generators for conditional distributions, constructing transition probability matrices for statistical or stochastic applications, and addressing various tasks relevant to the Artificial Intelligence community. The proposed algorithms are highly relevant to supervised and unsupervised learning, generative methods, as well as reinforcement learning.

A class of kernel-based scalable algorithms for data science

TL;DR

The paper addresses the scalability gap of kernel-based RKHS methods in data science by introducing a multi-scale divide-and-conquer framework that preserves reproducibility and error bounds. It advances a suite of general-purpose algorithms (greedy, permutation, explicit descent) and clustering strategies (sharp discrepancy sequences and balanced clustering) to enable efficient regression, interpolation, and optimal transport on large/high-dimensional datasets. It also offers encoder–decoder sampling and conditional distribution modeling to handle distribution-to-distribution mappings and conditional generation, with a unifying OT-like framework. Numerical experiments across clustering, MNIST classification, multiscale OT, conditional sampling, and the Bachelier test demonstrate practical performance and throughputs, supported by an open-source CodPy library for reproducible research.

Abstract

We present several generative and predictive algorithms based on the RKHS (reproducing kernel Hilbert spaces) methodology, which, most importantly, are scale up efficiently with large datasets or high-dimensional data. It is well recognized that the RKHS methodology leads one to efficient and robust algorithms for numerous tasks in data science, statistics, and scientific computation. However, the implementations existing the literature are often difficult to scale up for encompassing large datasets. In this paper, we introduce a simple and robust, divide-and-conquer methodology. It applies to large scale datasets and relies on several kernel-based algorithms, which distinguish between various extrapolation, interpolation, and optimal transport steps. We argue how to select the suitable algorithm in specific applications thanks to a feedback of performance criteria. Our primary focus is on applications and problems arising in industrial contexts, such as generating meshes for efficient numerical simulations, designing generators for conditional distributions, constructing transition probability matrices for statistical or stochastic applications, and addressing various tasks relevant to the Artificial Intelligence community. The proposed algorithms are highly relevant to supervised and unsupervised learning, generative methods, as well as reinforcement learning.

Paper Structure

This paper contains 33 sections, 39 equations, 10 figures, 5 algorithms.

Table of Contents

  1. Introduction
  2. Fundamentals on kernel methods
  3. Aim
  4. Framework for this paper
  5. Dealing with two distributions
  6. Multi-scale kernel methods
  7. General purpose algorithms
  8. Greedy search algorithms
  9. Permutation algorithms
  10. Linear sum assignment problems
  11. Discrete descent algorithms
  12. Explicit descent algorithms
  13. Clustering algorithms
  14. Proposed strategy
  15. Greedy clustering methods
  16. ...and 18 more sections

Figures (10)

  • Figure 1: Comparison of clustering methods
  • Figure 2: Performance of interpolations for a given kernel - MNIST problem
  • Figure 3: Performance of extrapolations for a given kernel - MNIST problem
  • Figure 4: Performance of multiscale learning for a given kernel - MNIST problem
  • Figure 5: Performance of LSAP vs Sinkhorn Optimal Transport
  • ...and 5 more figures