Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OptBA: Optimizing Hyperparameters with the Bees Algorithm for Improved Medical Text Classification

Mai A. Shaaban, Mariam Kashkash, Maryam Alghfeli, Adham Ibrahim

TL;DR

OptBA addresses hyperparameter tuning challenges in medical text classification by applying the Bees Algorithm to search two key LSTM hyperparameters (epochs and units) on an augmented Mooney dataset, achieving measurable accuracy gains over baselines. The method uses a population-based search with local and global exploration, and compares favorably with Optuna by mitigating local optima and enabling faster early convergence in some settings. The work demonstrates that modest hyperparameter refinements (e.g., increasing epochs and units) can yield meaningful improvements, including a $1.4\%$ lift for LSTM and up to $3.3\%$ for CNN on a 25-class medical text task. It provides an open-source implementation and highlights potential extensions to other domains and models in healthcare NLP.

Abstract

One of the main challenges in the field of deep learning is obtaining the optimal model hyperparameters. The search for optimal hyperparameters usually hinders the progress of solutions to real-world problems such as healthcare. Previous solutions have been proposed, but they can still get stuck in local optima. To overcome this hurdle, we propose OptBA to automatically fine-tune the hyperparameters of deep learning models by leveraging the Bees Algorithm, which is a recent promising swarm intelligence algorithm. In this paper, the optimization problem of OptBA is to maximize the accuracy in classifying ailments using medical text, where initial hyperparameters are iteratively adjusted by specific criteria. Experimental results demonstrate a noteworthy enhancement in accuracy with approximately 1.4%. This outcome highlights the effectiveness of the proposed mechanism in addressing the critical issue of hyperparameter optimization and its potential impact on advancing solutions for healthcare. The code is available publicly at \url{https://github.com/Mai-CS/OptBA}.

OptBA: Optimizing Hyperparameters with the Bees Algorithm for Improved Medical Text Classification

TL;DR

OptBA addresses hyperparameter tuning challenges in medical text classification by applying the Bees Algorithm to search two key LSTM hyperparameters (epochs and units) on an augmented Mooney dataset, achieving measurable accuracy gains over baselines. The method uses a population-based search with local and global exploration, and compares favorably with Optuna by mitigating local optima and enabling faster early convergence in some settings. The work demonstrates that modest hyperparameter refinements (e.g., increasing epochs and units) can yield meaningful improvements, including a lift for LSTM and up to for CNN on a 25-class medical text task. It provides an open-source implementation and highlights potential extensions to other domains and models in healthcare NLP.

Abstract

One of the main challenges in the field of deep learning is obtaining the optimal model hyperparameters. The search for optimal hyperparameters usually hinders the progress of solutions to real-world problems such as healthcare. Previous solutions have been proposed, but they can still get stuck in local optima. To overcome this hurdle, we propose OptBA to automatically fine-tune the hyperparameters of deep learning models by leveraging the Bees Algorithm, which is a recent promising swarm intelligence algorithm. In this paper, the optimization problem of OptBA is to maximize the accuracy in classifying ailments using medical text, where initial hyperparameters are iteratively adjusted by specific criteria. Experimental results demonstrate a noteworthy enhancement in accuracy with approximately 1.4%. This outcome highlights the effectiveness of the proposed mechanism in addressing the critical issue of hyperparameter optimization and its potential impact on advancing solutions for healthcare. The code is available publicly at \url{https://github.com/Mai-CS/OptBA}.
Paper Structure (11 sections, 1 figure, 2 tables, 1 algorithm)

This paper contains 11 sections, 1 figure, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Data Preprocessing
  5. The Bees Algorithm
  6. Hyperparameter Tuning using the Bees Algorithm
  7. Results and Discussion
  8. Model Configuration
  9. Comparison with Optuna
  10. Limitations
  11. Conclusion

Figures (1)

  • Figure 1: Class distribution of the ailments dataset.