Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TREB: a BERT attempt for imputing tabular data imputation

Shuyue Wang, Wenjun Zhou, Han drk-m-s Jiang, Shuo Wang, Ren Zheng

TL;DR

The paper comprehensively addresses the unique challenges posed by tabular data imputation, emphasizing the importance of context-based interconnections and its ability to preserve feature interrelationships and accurately impute missing values.

Abstract

TREB, a novel tabular imputation framework utilizing BERT, introduces a groundbreaking approach for handling missing values in tabular data. Unlike traditional methods that often overlook the specific demands of imputation, TREB leverages the robust capabilities of BERT to address this critical task. While many BERT-based approaches for tabular data have emerged, they frequently under-utilize the language model's full potential. To rectify this, TREB employs a BERT-based model fine-tuned specifically for the task of imputing real-valued continuous numbers in tabular datasets. The paper comprehensively addresses the unique challenges posed by tabular data imputation, emphasizing the importance of context-based interconnections. The effectiveness of TREB is validated through rigorous evaluation using the California Housing dataset. The results demonstrate its ability to preserve feature interrelationships and accurately impute missing values. Moreover, the authors shed light on the computational efficiency and environmental impact of TREB, quantifying the floating-point operations (FLOPs) and carbon footprint associated with its training and deployment.

TREB: a BERT attempt for imputing tabular data imputation

TL;DR

The paper comprehensively addresses the unique challenges posed by tabular data imputation, emphasizing the importance of context-based interconnections and its ability to preserve feature interrelationships and accurately impute missing values.

Abstract

TREB, a novel tabular imputation framework utilizing BERT, introduces a groundbreaking approach for handling missing values in tabular data. Unlike traditional methods that often overlook the specific demands of imputation, TREB leverages the robust capabilities of BERT to address this critical task. While many BERT-based approaches for tabular data have emerged, they frequently under-utilize the language model's full potential. To rectify this, TREB employs a BERT-based model fine-tuned specifically for the task of imputing real-valued continuous numbers in tabular datasets. The paper comprehensively addresses the unique challenges posed by tabular data imputation, emphasizing the importance of context-based interconnections. The effectiveness of TREB is validated through rigorous evaluation using the California Housing dataset. The results demonstrate its ability to preserve feature interrelationships and accurately impute missing values. Moreover, the authors shed light on the computational efficiency and environmental impact of TREB, quantifying the floating-point operations (FLOPs) and carbon footprint associated with its training and deployment.
Paper Structure (15 sections, 7 figures, 2 tables)

This paper contains 15 sections, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. BERT and tabular data
  4. TABERT
  5. TREB: TabluaR imputatEr using Bert.
  6. Data and tokenization
  7. Training paradigm
  8. Evaluation
  9. Hardware
  10. Performance
  11. Errors
  12. Flops consumption and carbon footprint
  13. Flops
  14. Carbon footprint
  15. Conclusion

Figures (7)

  • Figure 1: The three major types summarized in GenAI approaches in tabular data manipulation: 1. using diffusion model; 2 .using pretrained language model (e.g. GPT2); 3. using BERT (to which this paper belongs).
  • Figure 2: Workflow of TREB.
  • Figure 3: Training paradigm.
  • Figure 4: Imputation on column 0.
  • Figure 5: Imputation on column 1.
  • ...and 2 more figures