Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multiple Locally Linear Kernel Machines

David Picard

TL;DR

The paper introduces Multiple Locally Linear Kernel Machines (MLLKM), a non-linear classifier built from a large set of locally linear kernels assembled via an $ ext{L}_1$-MKL framework. It develops SequentialMKL to efficiently train with streaming kernels, yielding fast inference comparable to linear models while achieving non-linear accuracy similar to kernel methods. By formulating a primal and dual MKL problem and exploiting an explicit conformal feature mapping, MLLKM achieves sparsity and low memory usage, enabling deployment on large datasets. The approach is validated on synthetic and UCI datasets, showing favorable accuracy and substantially reduced inference time and kernel storage compared to traditional MKL and kernel SVM baselines.

Abstract

In this paper we propose a new non-linear classifier based on a combination of locally linear classifiers. A well known optimization formulation is given as we cast the problem in a $\ell_1$ Multiple Kernel Learning (MKL) problem using many locally linear kernels. Since the number of such kernels is huge, we provide a scalable generic MKL training algorithm handling streaming kernels. With respect to the inference time, the resulting classifier fits the gap between high accuracy but slow non-linear classifiers (such as classical MKL) and fast but low accuracy linear classifiers.

Multiple Locally Linear Kernel Machines

TL;DR

The paper introduces Multiple Locally Linear Kernel Machines (MLLKM), a non-linear classifier built from a large set of locally linear kernels assembled via an -MKL framework. It develops SequentialMKL to efficiently train with streaming kernels, yielding fast inference comparable to linear models while achieving non-linear accuracy similar to kernel methods. By formulating a primal and dual MKL problem and exploiting an explicit conformal feature mapping, MLLKM achieves sparsity and low memory usage, enabling deployment on large datasets. The approach is validated on synthetic and UCI datasets, showing favorable accuracy and substantially reduced inference time and kernel storage compared to traditional MKL and kernel SVM baselines.

Abstract

In this paper we propose a new non-linear classifier based on a combination of locally linear classifiers. A well known optimization formulation is given as we cast the problem in a Multiple Kernel Learning (MKL) problem using many locally linear kernels. Since the number of such kernels is huge, we provide a scalable generic MKL training algorithm handling streaming kernels. With respect to the inference time, the resulting classifier fits the gap between high accuracy but slow non-linear classifiers (such as classical MKL) and fast but low accuracy linear classifiers.
Paper Structure (16 sections, 25 equations, 2 figures, 2 tables, 2 algorithms)

This paper contains 16 sections, 25 equations, 2 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Locally Linear Classifiers
  3. Locally linear kernels
  4. Multiple Locally Linear Kernel Machines
  5. Sequential MKL
  6. Inference computational cost
  7. Parameter selection
  8. Experiments
  9. Synthetic data
  10. UCI dataset
  11. Conclusion
  12. ICML'15 reviews
  13. Reviewer 3
  14. Reviewer 4
  15. Reviewer 6
  16. ...and 1 more sections

Figures (2)

  • Figure 1: Example of synthetic data and corresponding $\mathcal{X}_c$ for different types of maps, with anchor point $a$.
  • Figure 2: Example of synthetic data classification task. The true boundary is shown in plain black line, while the classifier boundary is in dashed black line. The circles denote the support vectors, while the squares denote the support kernels.