Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Diagnostic Accuracy for Urinary Tract Disease through Explainable SHAP-Guided Feature Selection and Classification

Filipe Ferreira de Oliveira, Matheus Becali Rocha, Renato A. Krohling

TL;DR

This study addresses the need for accurate and explainable diagnosis of urinary tract diseases, focusing on bladder cancer. It proposes a SHAP-based feature selection framework integrated into a pipeline with SMOTE oversampling and Optuna-driven hyperparameter optimization across three gradient-boosting models (XGBoost, LightGBM, CatBoost). The SHAP-guided reduction often preserves or enhances balanced accuracy, precision, and specificity across six binary classification tasks, outperforming a prior baseline in many cases and yielding interpretable feature rankings aligned with clinical rationale. The approach enhances transparency in clinical decision support, identifying plausible biomarkers (e.g., Urine Epithelium, Gender) and supporting screening and early diagnosis. Overall, SHAP-guided feature selection within robust preprocessing and modeling pipelines shows promise for reliable, explainable clinical tools in urologic oncology.

Abstract

In this paper, we propose an approach to support the diagnosis of urinary tract diseases, with a focus on bladder cancer, using SHAP (SHapley Additive exPlanations)-based feature selection to enhance the transparency and effectiveness of predictive models. Six binary classification scenarios were developed to distinguish bladder cancer from other urological and oncological conditions. The algorithms XGBoost, LightGBM, and CatBoost were employed, with hyperparameter optimization performed using Optuna and class balancing with the SMOTE technique. The selection of predictive variables was guided by importance values through SHAP-based feature selection while maintaining or even improving performance metrics such as balanced accuracy, precision, and specificity. The use of explainability techniques (SHAP) for feature selection proved to be an effective approach. The proposed methodology may contribute to the development of more transparent, reliable, and efficient clinical decision support systems, optimizing screening and early diagnosis of urinary tract diseases.

Enhancing Diagnostic Accuracy for Urinary Tract Disease through Explainable SHAP-Guided Feature Selection and Classification

TL;DR

This study addresses the need for accurate and explainable diagnosis of urinary tract diseases, focusing on bladder cancer. It proposes a SHAP-based feature selection framework integrated into a pipeline with SMOTE oversampling and Optuna-driven hyperparameter optimization across three gradient-boosting models (XGBoost, LightGBM, CatBoost). The SHAP-guided reduction often preserves or enhances balanced accuracy, precision, and specificity across six binary classification tasks, outperforming a prior baseline in many cases and yielding interpretable feature rankings aligned with clinical rationale. The approach enhances transparency in clinical decision support, identifying plausible biomarkers (e.g., Urine Epithelium, Gender) and supporting screening and early diagnosis. Overall, SHAP-guided feature selection within robust preprocessing and modeling pipelines shows promise for reliable, explainable clinical tools in urologic oncology.

Abstract

In this paper, we propose an approach to support the diagnosis of urinary tract diseases, with a focus on bladder cancer, using SHAP (SHapley Additive exPlanations)-based feature selection to enhance the transparency and effectiveness of predictive models. Six binary classification scenarios were developed to distinguish bladder cancer from other urological and oncological conditions. The algorithms XGBoost, LightGBM, and CatBoost were employed, with hyperparameter optimization performed using Optuna and class balancing with the SMOTE technique. The selection of predictive variables was guided by importance values through SHAP-based feature selection while maintaining or even improving performance metrics such as balanced accuracy, precision, and specificity. The use of explainability techniques (SHAP) for feature selection proved to be an effective approach. The proposed methodology may contribute to the development of more transparent, reliable, and efficient clinical decision support systems, optimizing screening and early diagnosis of urinary tract diseases.
Paper Structure (17 sections, 5 equations, 2 figures, 3 tables)

This paper contains 17 sections, 5 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Material and Methods
  3. Patient data
  4. Data Preprocessing
  5. Missing Data Imputation
  6. Oversampling
  7. Dimensionality Reduction
  8. Algorithms
  9. Extreme Gradient Boosting (XGBoost)
  10. Categorical Boosting (CatBoost)
  11. Light Gradient Boosting Machine (LightGBM)
  12. Experimental Methodology
  13. Metrics
  14. Hyperparameters setting
  15. Results
  16. ...and 2 more sections

Figures (2)

  • Figure 1: General framework flow: the dataset is divided according to experiments and classes. The training set undergoes preprocessing (imputation, normalization, and encoding) and oversampling with SMOTE, followed by hyperparameter optimization using Optuna. With the optimal parameters, the model is trained and evaluated, generating performance metrics. Subsequently, SHAP values are used for dimensionality reduction and the selection of the most relevant features for classification.
  • Figure 2: SHAP values for the top 20 features of the dataset across all scenarios.