Improving Omics-Based Classification: The Role of Feature Selection and Synthetic Data Generation

TL;DR

The study addresses omics-based classification under limited sample size and interpretability constraints. It proposes a L1-regularized feature selection framework coupled with Gaussian-noise data augmentation and a kernel-based KSVM, evaluated on the E-MTAB-8026 miRNA dataset across six binary tasks via bootstrap analysis. Results show that synthetic data enhances generalization in small-sample settings and reduces feature dimensionality while maintaining competitive accuracy, offering improved interpretability over full LASSO models. The work highlights a trade-off between accuracy and explainability and suggests pursuing advanced generative augmentation to better capture biological variability.

Abstract

Given the increasing complexity of omics datasets, a key challenge is not only improving classification performance but also enhancing the transparency and reliability of model decisions. Effective model performance and feature selection are fundamental for explainability and reliability. In many cases, high dimensional omics datasets suffer from limited number of samples due to clinical constraints, patient conditions, phenotypes rarity and others conditions. Current omics based classification models often suffer from narrow interpretability, making it difficult to discern meaningful insights where trust and reproducibility are critical. This study presents a machine learning based classification framework that integrates feature selection with data augmentation techniques to achieve high standard classification accuracy while ensuring better interpretability. Using the publicly available dataset (E MTAB 8026), we explore a bootstrap analysis in six binary classification scenarios to evaluate the proposed model's behaviour. We show that the proposed pipeline yields cross validated perfomance on small dataset that is conserved when the trained classifier is applied to a larger test set. Our findings emphasize the fundamental balance between accuracy and feature selection, highlighting the positive effect of introducing synthetic data for better generalization, even in scenarios with very limited samples availability.

Figures (2)

• Figure 1: Experimental Procedure for the Bootstrap Analysis. The preprocessed dataset is divided into six binary classification scenarios. For each scenario, training set sampling is performed at different sizes, while the remaining samples constitute the test set. Three classification approaches are evaluated: (1) the L1-KSVM Framework with augmentation, where 200 synthetic samples per class are generated using a Gaussian noise-based augmentation technique, (2) the L1-KSVM Framework without augmentation, which follows the same feature selection and classification procedure without synthetic data, and (3) the Baseline LASSO regression model, where features are retained if their L1-coefficient is greater than zero. Features collection and classification is then performed. Resulting data are collected to conduct performance analysis.
• Figure 2: Each subplot corresponds to a binary classification scenario: Scenario 1 (Healthy Controls vs. LCa), Scenario 2 (Healthy Controls vs. NTLD), Scenario 3 (Healthy Controls vs. OD), Scenario 4 (LCa vs. NTLD), Scenario 5 (LCa vs. OD), and Scenario 6 (NTLD vs. OD). The x-axis represents the training sample size, while the y-axis shows the classification accuracy. The plots compare three approaches: the L1-KSVM Framework with Data Augmentation (blue solid line), incorporating 200 synthetic samples per class; the L1-KSVM Framework without Augmentation (red solid line), applying the same feature selection and classification process but without synthetic data; and the Baseline LASSO Regression Model (green dashed line), which retains all features with a nonzero L1 coefficient. Shaded areas indicate standard deviation across 100 bootstrap iterations per sample size.