Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

$L_1$-norm Regularized Indefinite Kernel Logistic Regression

Shaoxin Wang, Hanjing Yao

TL;DR

A novel $L_1$-norm regularized indefinite kernel logistic regression model, which extends the existing IKLR framework by introducing sparsity via an $L_1$-norm penalty, which enhances interpretability and generalization while introducing nonsmoothness and nonconvexity into the optimization landscape.

Abstract

Kernel logistic regression (KLR) is a powerful classification method widely applied across diverse domains. In many real-world scenarios, indefinite kernels capture more domain-specific structural information than positive definite kernels. This paper proposes a novel $L_1$-norm regularized indefinite kernel logistic regression (RIKLR) model, which extends the existing IKLR framework by introducing sparsity via an $L_1$-norm penalty. The introduction of this regularization enhances interpretability and generalization while introducing nonsmoothness and nonconvexity into the optimization landscape. To address these challenges, a theoretically grounded and computationally efficient proximal linearized algorithm is developed. Experimental results on multiple benchmark datasets demonstrate the superior performance of the proposed method in terms of both accuracy and sparsity.

$L_1$-norm Regularized Indefinite Kernel Logistic Regression

TL;DR

A novel -norm regularized indefinite kernel logistic regression model, which extends the existing IKLR framework by introducing sparsity via an -norm penalty, which enhances interpretability and generalization while introducing nonsmoothness and nonconvexity into the optimization landscape.

Abstract

Kernel logistic regression (KLR) is a powerful classification method widely applied across diverse domains. In many real-world scenarios, indefinite kernels capture more domain-specific structural information than positive definite kernels. This paper proposes a novel -norm regularized indefinite kernel logistic regression (RIKLR) model, which extends the existing IKLR framework by introducing sparsity via an -norm penalty. The introduction of this regularization enhances interpretability and generalization while introducing nonsmoothness and nonconvexity into the optimization landscape. To address these challenges, a theoretically grounded and computationally efficient proximal linearized algorithm is developed. Experimental results on multiple benchmark datasets demonstrate the superior performance of the proposed method in terms of both accuracy and sparsity.

Paper Structure

This paper contains 14 sections, 3 theorems, 45 equations, 1 figure, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. A brief review of related KLR models
  3. The KLR model
  4. The $L_1$-norm RKLR model
  5. The IKLR model
  6. The RIKLR model
  7. The $L_1$-norm RIKLR model
  8. Optimization algorithm
  9. Convergence analysis
  10. Numerical experiment
  11. Experimental setup
  12. Results on Kaggle and UCI Databases
  13. Results on Mnist Database
  14. Concluding remark

Key Result

Lemma 2.1

In the RKKS, there exists a positive decomposition of the indefinite kernel $\mathcal{K}{\boldsymbol{(x_i, x_j)}}$ such that where $\mathcal{K}_+$ and $\mathcal{K}_-$ are two PD kernels.

Figures (1)

  • Figure 1: Some examples from the Mnist Database.

Theorems & Definitions (7)

  • Definition 1: Bogn12
  • Lemma 2.1: OMCS04
  • Remark 1
  • Remark 2
  • Theorem 3.1
  • Theorem 3.2
  • Remark 3