Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Random Fourier Signature Features

Csaba Toth, Harald Oberhauser, Zoltan Szabo

TL;DR

This work develops a random Fourier feature-based acceleration of the signature kernel acting on the inherently non-Euclidean domain of sequences, and derives two even more scalable time series features with favourable concentration properties and computational complexity both in time and memory.

Abstract

Tensor algebras give rise to one of the most powerful measures of similarity for sequences of arbitrary length called the signature kernel accompanied with attractive theoretical guarantees from stochastic analysis. Previous algorithms to compute the signature kernel scale quadratically in terms of the length and the number of the sequences. To mitigate this severe computational bottleneck, we develop a random Fourier feature-based acceleration of the signature kernel acting on the inherently non-Euclidean domain of sequences. We show uniform approximation guarantees for the proposed unbiased estimator of the signature kernel, while keeping its computation linear in the sequence length and number. In addition, combined with recent advances on tensor projections, we derive two even more scalable time series features with favourable concentration properties and computational complexity both in time and memory. Our empirical results show that the reduction in computational cost comes at a negligible price in terms of accuracy on moderate-sized datasets, and it enables one to scale to large datasets up to a million time series.

Random Fourier Signature Features

TL;DR

This work develops a random Fourier feature-based acceleration of the signature kernel acting on the inherently non-Euclidean domain of sequences, and derives two even more scalable time series features with favourable concentration properties and computational complexity both in time and memory.

Abstract

Tensor algebras give rise to one of the most powerful measures of similarity for sequences of arbitrary length called the signature kernel accompanied with attractive theoretical guarantees from stochastic analysis. Previous algorithms to compute the signature kernel scale quadratically in terms of the length and the number of the sequences. To mitigate this severe computational bottleneck, we develop a random Fourier feature-based acceleration of the signature kernel acting on the inherently non-Euclidean domain of sequences. We show uniform approximation guarantees for the proposed unbiased estimator of the signature kernel, while keeping its computation linear in the sequence length and number. In addition, combined with recent advances on tensor projections, we derive two even more scalable time series features with favourable concentration properties and computational complexity both in time and memory. Our empirical results show that the reduction in computational cost comes at a negligible price in terms of accuracy on moderate-sized datasets, and it enables one to scale to large datasets up to a million time series.
Paper Structure (39 sections, 26 theorems, 158 equations, 1 figure, 3 tables, 3 algorithms)

This paper contains 39 sections, 26 theorems, 158 equations, 1 figure, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Features vs Kernel/Primal vs Dual
  3. Computational Cost of the Signature kernel
  4. Contribution
  5. Related Work
  6. Outline
  7. Prerequisites
  8. Notation
  9. Random Fourier Features
  10. Tensors and the tensor product
  11. Free algebras
  12. Path Signatures
  13. Signature Features for Sequential Data
  14. Signature Kernels
  15. Random Fourier Signature Features
  16. ...and 24 more sections

Key Result

Theorem 3.2

\newlabelthm:main0 Let $\texttt{k}: \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}$ be a continuous, bounded, translation-invariant kernel with spectral measure $\Lambda$, which satisfies eq:w_cond_main. Let $\mathcal{X} \subset \mathbb{R}^d$ be compact and convex with diameter $\left\lvert \math where $C_{d, \mathcal{X}} \coloneqq 2^\frac{1}{d+1} 16 \left\lvert \mathcal{X} \right\rvert^\frac{d}

Figures (1)

  • Figure 1: Approximation error of random kernels against RFF sample size on $\log$-$\log$ plot.

Theorems & Definitions (59)

  • Remark 2.1
  • Definition 3.1
  • Theorem 3.2
  • Remark 3.3
  • Definition 3.4
  • Theorem 3.5
  • Remark 3.6
  • Definition 3.7
  • Theorem 3.8
  • Remark 3.9
  • ...and 49 more