Comparative Analysis of Data Preprocessing Methods, Feature Selection Techniques and Machine Learning Models for Improved Classification and Regression Performance on Imbalanced Genetic Data

TL;DR

This study investigates how data preprocessing, feature selection, and model choice affect predictive performance on imbalanced genetic datasets with high-cardinality features. Using multiple transformations and model-selection strategies across regression (CADD_PHRED) and classification (SIFT, PolyPhen) targets, the authors find that random forest consistently offers strong performance for imbalanced regression, while classification benefits are less pronounced and can be achieved with RF in many configurations; log-transformations often degrade performance and dropping outliers provides only marginal gains. The results suggest robustness of tree-based methods to data skew and imbalance in this domain, with findings that generalize to similar high-dimensional, skewed genomic data. The work also highlights challenges in statistically isolating the best feature-selection approach and points to future directions such as advanced optimization (e.g., SERA) and broader validation.

Abstract

Rapid advancements in genome sequencing have led to the collection of vast amounts of genomics data. Researchers may be interested in using machine learning models on such data to predict the pathogenicity or clinical significance of a genetic mutation. However, many genetic datasets contain imbalanced target variables that pose challenges to machine learning models: observations are skewed/imbalanced in regression tasks or class-imbalanced in classification tasks. Genetic datasets are also often high-cardinal and contain skewed predictor variables, which poses further challenges. We aimed to investigate the effects of data preprocessing, feature selection techniques, and model selection on the performance of models trained on these datasets. We measured performance with 5-fold cross-validation and compared averaged r-squared and accuracy metrics across different combinations of techniques. We found that outliers/skew in predictor or target variables did not pose a challenge to regression models. We also found that class-imbalanced target variables and skewed predictors had little to no impact on classification performance. Random forest was the best model to use for imbalanced regression tasks. While our study uses a genetic dataset as an example of a real-world application, our findings can be generalized to any similar datasets.

Figures (5)

• Figure 1: Workflow of Data Preprocessing: The dataset used in this research, clinvar_cleaned.csv was obtained from the authors of Kaur2024PredictingLI, who cleaned an original dataset from kevin_arvai_2020. The process of creating the cleaned dataset (removing conflicting information, dropping irrelevantlow-variance variables, and fixing errors in data entry) is described in sections 2.1 to 2.3 of Kaur2024PredictingLI. The rest of this figure details the datasets created in this study.
• Figure 2: SIFT and Polyphen Class Distribution: In this figure, we visualized SIFT and PolyPhen class distributions in order to assess if the class-imbalance problem was present in each dataset guo.
• Figure 3: CADD_PHRED's Heavy Right Skew: The presence of outliers and heavy right skew in CADD_PHRED indicates that models training on this data have to deal with the imbalanced regression problem, which is explained in greater detail in the introduction section.
• Figure 4: Comparing Continuous Variables Across Datasets: We found six variables (AF_ESP, AF_EXAC,  AF_TGP, cDNA_position, CDS_position, Protein_position) that were heavily skewed in all three datasets (df_polyphen.csv, df_sift.csv, and df_caddphred.csv for the top, middle and bottom panels respectively).
• Figure 5: Examples of Combinations of Dataset, Feature Selection Techniques, and Model Choices: Each of the twelve datasets from Sets 1 through 4 (see Tables 1 and 2) produced eleven unique combinations of the dataset used, feature selection technique applied, and model trained. Three feature sets were created for each dataset using different univariate feature selection techniques, and three models were trained for each feature set. In addition, for every dataset, two more models, random forest and decision trees were trained with no specified feature set, allowing the models to use their embedded feature selection techniques. In total, there were twelve datasets in Sets 1-4 and eleven combinations for each, meaning we tested one hundred and thirty-two total combinations. Results for the regressors were saved to one file and classifiers to another.