One Class Restricted Kernel Machines

TL;DR

The paper addresses robust one-class anomaly detection within kernel-based models by introducing OCRKM, an RBM-inspired energy formulation integrated into the Restricted Kernel Machine framework. Using the Fenchel-Young inequality, OCRKM derives a tractable, kernelized objective that couples visible projections $\xi$ with hidden features $h$, leading to a dual decision rule $f(x) = \frac{1}{\gamma} \sum_i h_i K(x_i, x) - \rho$. Empirically, OCRKM outperforms strong baselines on 30 UCI datasets, achieving higher average accuracy and showing statistically significant improvements via Friedman and Nemenyi tests. The work demonstrates enhanced robustness against contamination and outlines future directions toward multiview extensions and reduced computational demands.

Abstract

Restricted kernel machines (RKMs) have demonstrated a significant impact in enhancing generalization ability in the field of machine learning. Recent studies have introduced various methods within the RKM framework, combining kernel functions with the least squares support vector machine (LSSVM) in a manner similar to the energy function of restricted boltzmann machines (RBM), such that a better performance can be achieved. However, RKM's efficacy can be compromised by the presence of outliers and other forms of contamination within the dataset. These anomalies can skew the learning process, leading to less accurate and reliable outcomes. To address this critical issue and to ensure the robustness of the model, we propose the novel one-class RKM (OCRKM). In the framework of OCRKM, we employ an energy function akin to that of the RBM, which integrates both visible and hidden variables in a nonprobabilistic setting. The formulation of the proposed OCRKM facilitates the seamless integration of one-class classification method with the RKM, enhancing its capability to detect outliers and anomalies effectively. The proposed OCRKM model is evaluated over UCI benchmark datasets. Experimental findings and statistical analyses consistently emphasize the superior generalization capabilities of the proposed OCRKM model over baseline models across all scenarios.

Figures (2)

• Figure 1: The geometrical structure of the One-Class Robust Kernel Machine (OCRKM) model involves mapping an input $x$ from the original space to a higher-dimensional feature space using a non-linear mapping function $\phi(x)$. In this feature space, the model learns projections $\xi$ that capture the underlying structure of the data while being coupled with hidden features $h$ in a latent space. These hidden features $h$ help to model the intrinsic characteristics of the data distribution, enabling the model to distinguish between normal and abnormal data points. The coupling between the feature space projections $\xi$ and the latent space features $h$ allows the OCRKM model to effectively learn robust decision boundaries that are less sensitive to noise and outliers, improving its one-class classification performance.
• Figure 2: Effect of hyperparameters $\eta$ and $\gamma$ on the performance of the proposed OCRKM model.