Invertible Kernel PCA with Random Fourier Features

TL;DR

This paper presents an alternative method to reconstruct the original input signal from kPCA where the reconstruction follows naturally from the compression step, and exploits the fact that the nonlinear transformation is invertible in a certain subdomain.

Abstract

Kernel principal component analysis (kPCA) is a widely studied method to construct a low-dimensional data representation after a nonlinear transformation. The prevailing method to reconstruct the original input signal from kPCA -- an important task for denoising -- requires us to solve a supervised learning problem. In this paper, we present an alternative method where the reconstruction follows naturally from the compression step. We first approximate the kernel with random Fourier features. Then, we exploit the fact that the nonlinear transformation is invertible in a certain subdomain. Hence, the name \emph{invertible kernel PCA (ikPCA)}. We experiment with different data modalities and show that ikPCA performs similarly to kPCA with supervised reconstruction on denoising tasks, making it a strong alternative.

Figures (13)

• Figure 1: Illustration of our invertible kernel PCA method.
• Figure 2: S-curve toy example. Reconstruction MSE for a different number of random features chosen for ikPCA.
• Figure 3: USPS data. Effect of regularization parameter $\lambda$.
• Figure 4: USPS reconstruction.
• Figure 5: Denoising of ECG beats from lead I. The blue area marks the min/max values of the 21 test beats. The red dashed lines show all test reconstructions with ikPCA.