Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conformal Transformation of Kernels: A Geometric Perspective on Text Classification

Ioana Rădulescu, Alexandra Băicoianu, Adela Mihai

TL;DR

This work investigates conformal transformations of kernels within SVMs for high-dimensional text classification. It introduces a Gaussian Cosine kernel tailored to multinomial geometry and applies two conformal transformations (based on cosine and spherical geodesic distances) to improve class separability, particularly for suboptimal kernels. Experiments on the Reuters dataset across one-vs-rest and one-vs-one tasks show that conformal transforms can yield substantial gains in certain settings (e.g., Gaussian and GC kernels), and can reduce model complexity via fewer support vectors, though they do not consistently surpass the best original kernels. The study highlights the geometric interpretation of kernels on multinomial manifolds, offers a practical protocol to exploit these transforms, and suggests future work in alternative distance measures and representation schemes for further gains in text classification.

Abstract

In this article we investigate the effects of conformal transformations on kernel functions used in Support Vector Machines. Our focus lies in the task of text document categorization, which involves assigning each document to a particular category. We introduce a new Gaussian Cosine kernel alongside two conformal transformations. Building upon previous studies that demonstrated the efficacy of conformal transformations in increasing class separability on synthetic and low-dimensional datasets, we extend this analysis to the high-dimensional domain of text data. Our experiments, conducted on the Reuters dataset on two types of binary classification tasks, compare the performance of Linear, Gaussian, and Gaussian Cosine kernels against their conformally transformed counterparts. The findings indicate that conformal transformations can significantly improve kernel performance, particularly for sub-optimal kernels. Specifically, improvements were observed in 60% of the tested scenarios for the Linear kernel, 84% for the Gaussian kernel, and 80% for the Gaussian Cosine kernel. In light of these findings, it becomes clear that conformal transformations play a pivotal role in enhancing kernel performance, offering substantial benefits.

Conformal Transformation of Kernels: A Geometric Perspective on Text Classification

TL;DR

This work investigates conformal transformations of kernels within SVMs for high-dimensional text classification. It introduces a Gaussian Cosine kernel tailored to multinomial geometry and applies two conformal transformations (based on cosine and spherical geodesic distances) to improve class separability, particularly for suboptimal kernels. Experiments on the Reuters dataset across one-vs-rest and one-vs-one tasks show that conformal transforms can yield substantial gains in certain settings (e.g., Gaussian and GC kernels), and can reduce model complexity via fewer support vectors, though they do not consistently surpass the best original kernels. The study highlights the geometric interpretation of kernels on multinomial manifolds, offers a practical protocol to exploit these transforms, and suggests future work in alternative distance measures and representation schemes for further gains in text classification.

Abstract

In this article we investigate the effects of conformal transformations on kernel functions used in Support Vector Machines. Our focus lies in the task of text document categorization, which involves assigning each document to a particular category. We introduce a new Gaussian Cosine kernel alongside two conformal transformations. Building upon previous studies that demonstrated the efficacy of conformal transformations in increasing class separability on synthetic and low-dimensional datasets, we extend this analysis to the high-dimensional domain of text data. Our experiments, conducted on the Reuters dataset on two types of binary classification tasks, compare the performance of Linear, Gaussian, and Gaussian Cosine kernels against their conformally transformed counterparts. The findings indicate that conformal transformations can significantly improve kernel performance, particularly for sub-optimal kernels. Specifically, improvements were observed in 60% of the tested scenarios for the Linear kernel, 84% for the Gaussian kernel, and 80% for the Gaussian Cosine kernel. In light of these findings, it becomes clear that conformal transformations play a pivotal role in enhancing kernel performance, offering substantial benefits.
Paper Structure (20 sections, 5 theorems, 60 equations, 12 tables)

This paper contains 20 sections, 5 theorems, 60 equations, 12 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Support Vector Machines
  4. Geometry of the SVM
  5. Geometry of the Multinomial Distributions
  6. Proposed Method
  7. Gaussian Cosine Kernel
  8. Conformal Transformation
  9. Induced Riemannian Metric
  10. Metric Induced by the Conformal Transformation
  11. Experimental Procedure
  12. Experiments
  13. Text Documents Representation
  14. Dataset
  15. Data preparation
  16. ...and 5 more sections

Key Result

Theorem 2.1

Let $\mathcal{X}$ be a subset of $\mathbb R^n$, $\mathcal{X}\subset \mathbb{R}^n$. The function $k : \mathcal{X}^2 \to \mathbb R$ is a Mercer kernel function (also known as kernel function) if: The corresponding kernel matrix of a Mercer kernel is called a Mercer kernel matrix.

Theorems & Definitions (13)

  • Theorem 2.1
  • Proposition 2.2
  • Proposition 2.3
  • proof
  • Proposition 2.4
  • proof
  • Definition 2.5
  • Definition 2.6
  • Definition 2.7
  • Definition 3.1
  • ...and 3 more