Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Geodesic Flow Kernels for Semi-Supervised Learning on Mixed-Variable Tabular Dataset

Yoontae Hwang, Yongjae Lee

TL;DR

This work addresses semi-supervised learning on real-world tabular data with mixed continuous and categorical features, where traditional methods struggle to capture structure. The authors introduce GFTab, which combines variable-specific corruption, a geodesic flow kernel defined on the Grassmannian to quantify similarity between corrupted views, and a tree-based embedding to leverage labeled data. On 21 tabular benchmarks, GFTab demonstrates strong performance, especially under limited labeled data and label noise, outperforming several strong baselines in many settings. The results underscore the value of geometry-aware similarity and heterogeneous variable treatment for robust tabular learning with practical impact across domains.

Abstract

Tabular data poses unique challenges due to its heterogeneous nature, combining both continuous and categorical variables. Existing approaches often struggle to effectively capture the underlying structure and relationships within such data. We propose GFTab (Geodesic Flow Kernels for Semi- Supervised Learning on Mixed-Variable Tabular Dataset), a semi-supervised framework specifically designed for tabular datasets. GFTab incorporates three key innovations: 1) Variable-specific corruption methods tailored to the distinct properties of continuous and categorical variables, 2) A Geodesic flow kernel based similarity measure to capture geometric changes between corrupted inputs, and 3) Tree-based embedding to leverage hierarchical relationships from available labeled data. To rigorously evaluate GFTab, we curate a comprehensive set of 21 tabular datasets spanning various domains, sizes, and variable compositions. Our experimental results show that GFTab outperforms existing ML/DL models across many of these datasets, particularly in settings with limited labeled data.

Geodesic Flow Kernels for Semi-Supervised Learning on Mixed-Variable Tabular Dataset

TL;DR

This work addresses semi-supervised learning on real-world tabular data with mixed continuous and categorical features, where traditional methods struggle to capture structure. The authors introduce GFTab, which combines variable-specific corruption, a geodesic flow kernel defined on the Grassmannian to quantify similarity between corrupted views, and a tree-based embedding to leverage labeled data. On 21 tabular benchmarks, GFTab demonstrates strong performance, especially under limited labeled data and label noise, outperforming several strong baselines in many settings. The results underscore the value of geometry-aware similarity and heterogeneous variable treatment for robust tabular learning with practical impact across domains.

Abstract

Tabular data poses unique challenges due to its heterogeneous nature, combining both continuous and categorical variables. Existing approaches often struggle to effectively capture the underlying structure and relationships within such data. We propose GFTab (Geodesic Flow Kernels for Semi- Supervised Learning on Mixed-Variable Tabular Dataset), a semi-supervised framework specifically designed for tabular datasets. GFTab incorporates three key innovations: 1) Variable-specific corruption methods tailored to the distinct properties of continuous and categorical variables, 2) A Geodesic flow kernel based similarity measure to capture geometric changes between corrupted inputs, and 3) Tree-based embedding to leverage hierarchical relationships from available labeled data. To rigorously evaluate GFTab, we curate a comprehensive set of 21 tabular datasets spanning various domains, sizes, and variable compositions. Our experimental results show that GFTab outperforms existing ML/DL models across many of these datasets, particularly in settings with limited labeled data.

Paper Structure

This paper contains 20 sections, 12 equations, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Methodology
  4. The GFTab Framework I : Corruption Process
  5. The GFTab Framework II: Geodesic Flow Kernel
  6. The GFTab Framework III: Tree-Based Embedding
  7. Tabular Benchmarks Dataset
  8. Experiment
  9. Implementation details
  10. Is GFTab really effective for tabular datasets?
  11. How to corrupt categorical variables effectively?
  12. Is geodesic flow useful for tabular datasets?
  13. How to choose the balance between $\mathcal{L}_{\text{ce}}$ and $\mathcal{L}_{\text{sim}}$?
  14. Conclusion
  15. Acknowledgments
  16. ...and 5 more sections

Figures (4)

  • Figure 1: The proposed model (GFTab) is a semi-supervised framework specifically designed for handling tabular data.
  • Figure 2: Win matrices for different categorical variable corruption methods.
  • Figure 3: Win matrix between GFTab with three different similarity losses.
  • Figure 4: Performance of GFTab with different levels of balance parameter