Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CI-RKM: A Class-Informed Approach to Robust Restricted Kernel Machines

Ritik Mishra, Mushir Akhtar, M. Tanveer

TL;DR

CI-RKM addresses robustness gaps in Restricted Kernel Machines by introducing a class-informed weighting scheme that modulates sample contribution based on proximity to class centroids. By embedding a diagonal weight matrix $D$ into weighted conjugate feature duality and applying the Schur complement, CI-RKM yields a robust linear system that enhances generalization in noisy settings. Empirical results on 26 UCI datasets show CI-RKM achieving the highest average accuracy and statistically significant improvements over baselines, with strong resilience in ablation tests under label noise. The work advances kernel-based learning by integrating geometric class structure into the learning objective, offering a practical and scalable approach to robust classification.

Abstract

Restricted kernel machines (RKMs) represent a versatile and powerful framework within the kernel machine family, leveraging conjugate feature duality to address a wide range of machine learning tasks, including classification, regression, and feature learning. However, their performance can degrade significantly in the presence of noise and outliers, which compromises robustness and predictive accuracy. In this paper, we propose a novel enhancement to the RKM framework by integrating a class-informed weighted function. This weighting mechanism dynamically adjusts the contribution of individual training points based on their proximity to class centers and class-specific characteristics, thereby mitigating the adverse effects of noisy and outlier data. By incorporating weighted conjugate feature duality and leveraging the Schur complement theorem, we introduce the class-informed restricted kernel machine (CI-RKM), a robust extension of the RKM designed to improve generalization and resilience to data imperfections. Experimental evaluations on benchmark datasets demonstrate that the proposed CI-RKM consistently outperforms existing baselines, achieving superior classification accuracy and enhanced robustness against noise and outliers. Our proposed method establishes a significant advancement in the development of kernel-based learning models, addressing a core challenge in the field.

CI-RKM: A Class-Informed Approach to Robust Restricted Kernel Machines

TL;DR

CI-RKM addresses robustness gaps in Restricted Kernel Machines by introducing a class-informed weighting scheme that modulates sample contribution based on proximity to class centroids. By embedding a diagonal weight matrix into weighted conjugate feature duality and applying the Schur complement, CI-RKM yields a robust linear system that enhances generalization in noisy settings. Empirical results on 26 UCI datasets show CI-RKM achieving the highest average accuracy and statistically significant improvements over baselines, with strong resilience in ablation tests under label noise. The work advances kernel-based learning by integrating geometric class structure into the learning objective, offering a practical and scalable approach to robust classification.

Abstract

Restricted kernel machines (RKMs) represent a versatile and powerful framework within the kernel machine family, leveraging conjugate feature duality to address a wide range of machine learning tasks, including classification, regression, and feature learning. However, their performance can degrade significantly in the presence of noise and outliers, which compromises robustness and predictive accuracy. In this paper, we propose a novel enhancement to the RKM framework by integrating a class-informed weighted function. This weighting mechanism dynamically adjusts the contribution of individual training points based on their proximity to class centers and class-specific characteristics, thereby mitigating the adverse effects of noisy and outlier data. By incorporating weighted conjugate feature duality and leveraging the Schur complement theorem, we introduce the class-informed restricted kernel machine (CI-RKM), a robust extension of the RKM designed to improve generalization and resilience to data imperfections. Experimental evaluations on benchmark datasets demonstrate that the proposed CI-RKM consistently outperforms existing baselines, achieving superior classification accuracy and enhanced robustness against noise and outliers. Our proposed method establishes a significant advancement in the development of kernel-based learning models, addressing a core challenge in the field.

Paper Structure

This paper contains 12 sections, 2 theorems, 23 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Restricted Kernel Machine (RKM)
  4. Proposed Work
  5. Class-Informed Weighted Function
  6. Class Informed Restricted Kernel Machine (CI-RKM)
  7. Experiments, Results and Discussion
  8. Models Compared
  9. Experimental Setup
  10. Performance Evaluation and Statistical Analysis on UCI Datasets
  11. Ablation Study
  12. Conclusions

Key Result

Theorem 3.1

ha2013support Let $K(x, x')$ be a kernel function. Then the inner product distance is given by:

Figures (1)

  • Figure 1: Subfigure (a) illustrates the traditional RKM, which uses a standard kernel function for mapping input data into a higher-dimensional space. In contrast, subfigure (b) presents the proposed CI-RKM, where the conjugate features are modified by incorporating class-specific information, enhancing the model's ability to capture class distinctions in the data.

Theorems & Definitions (4)

  • Theorem 3.1
  • proof
  • Theorem 3.2
  • proof