Optimization of Transformer heart disease prediction model based on particle swarm optimization algorithm

TL;DR

This work tackles heart disease prediction by combining particle swarm optimization with a Transformer model to optimize hyperparameters and architecture. The PSO-augmented Transformer achieves 96.5% accuracy, outperforming a baseline Random Forest at 92.2% (a 4.3 percentage-point improvement) on an open-source dataset of 1888 samples. The approach leverages PSO to tune learning rate, depth, hidden size, and attention heads, within a standard encoder-decoder Transformer framework that employs self-attention and positional encoding. The results suggest that PSO-empowered Transformers can offer substantial accuracy gains for clinical prediction tasks, with implications for faster, more reliable health risk assessment and resource optimization.

Abstract

Aiming at the latest particle swarm optimization algorithm, this paper proposes an improved Transformer model to improve the accuracy of heart disease prediction and provide a new algorithm idea. We first use three mainstream machine learning classification algorithms - decision tree, random forest and XGBoost, and then output the confusion matrix of these three models. The results showed that the random forest model had the best performance in predicting the classification of heart disease, with an accuracy of 92.2%. Then, we apply the Transformer model based on particle swarm optimization (PSO) algorithm to the same dataset for classification experiment. The results show that the classification accuracy of the model is as high as 96.5%, 4.3 percentage points higher than that of random forest, which verifies the effectiveness of PSO in optimizing Transformer model. From the above research, we can see that particle swarm optimization significantly improves Transformer performance in heart disease prediction. Improving the ability to predict heart disease is a global priority with benefits for all humankind. Accurate prediction can enhance public health, optimize medical resources, and reduce healthcare costs, leading to healthier populations and more productive societies worldwide. This advancement paves the way for more efficient health management and supports the foundation of a healthier, more resilient global community.

Figures (9)

• Figure 1: Correlation heat maps.
• Figure 2: The schematic diagram of particle swarm optimization.
• Figure 3: The schematic diagram of Transformer.
• Figure 4: The optimization process.
• Figure 5: The confusion matrix of the decision tree model.