Provably Robust Pre-Trained Ensembles for Biomarker-Based Cancer Classification
Chongmin Lee, Jihie Kim
TL;DR
This work tackles the challenge of robust biomarker-based cancer classification under severe class imbalance by leveraging a pre-trained, meta-trained Hyperfast model and a simple three-model hard-vote ensemble with XGBoost and LightGBM. The authors show that using a small, PCA-based feature set (as few as $200$–$500$ components) can achieve state-of-the-art performance on TCGA tissue biomarkers across binary and multiclass tasks, while theoretically justifying robustness to priors and imbalances via a prototype-classifier margin and error-bounded hard voting. They further demonstrate generalization to external liquid-biopsy RNA cohorts, where the ensemble maintains reliability even with heterogeneous data and fewer features. Together, these results indicate that pre-trained tabular models plus minimal-tuning ensembling can deliver high accuracy and improved minority-class performance with limited features, and they provide theoretical insights into why this robustness emerges. The practical impact lies in scalable, cost-effective cancer screening pipelines that can adapt to distributional shifts and imbalanced data in real-world clinical settings.
Abstract
Certain cancer types, notably pancreatic cancer, are difficult to detect at an early stage, motivating robust biomarker-based screening. Liquid biopsies enable non-invasive monitoring of circulating biomarkers, but typical machine learning pipelines for high-dimensional tabular data (e.g., random forests, SVMs) rely on expensive hyperparameter tuning and can be brittle under class imbalance. We leverage a meta-trained Hyperfast model for classifying cancer, accomplishing the highest AUC of 0.9929 and simultaneously achieving robustness especially on highly imbalanced datasets compared to other ML algorithms in several binary classification tasks (e.g. breast invasive carcinoma; BRCA vs. non-BRCA). We also propose a novel ensemble model combining pre-trained Hyperfast model, XGBoost, and LightGBM for multi-class classification tasks, achieving an incremental increase in accuracy (0.9464) while merely using 500 PCA features; distinguishable from previous studies where they used more than 2,000 features for similar results. Crucially, we demonstrate robustness under class imbalance: empirically via balanced accuracy and minority-class recall across cancer-vs.-noncancer and cancer-vs.-rest settings, and theoretically by showing (i) a prototype-form final layer for Hyperfast that yields prior-insensitive decisions under bounded bias, and (ii) minority-error reductions for majority vote under mild error diversity. Together, these results indicate that pre-trained tabular models and simple ensembling can deliver state-of-the-art accuracy and improved minority-class performance with far fewer features and no additional tuning.