Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Constrained Hybrid Metaheuristic Algorithm for Probabilistic Neural Networks Learning

Piotr A. Kowalski, Szymon Kucharczyk, Jacek Mańdziuk

TL;DR

This work tackles training Probabilistic Neural Networks (PNNs) in uncertain, high-dimensional settings by proposing a constrained Hybrid Metaheuristic (cHM) that dynamically blends multiple swarm-based optimisers. The method operates in two phases—probing to select the best performing metaheuristic by minimizing the error rate, then fitting to refine the smoothing parameters within the KDE framework—achieving faster convergence and improved generalisation. Empirical results on 16 diverse datasets show cHM often outperforms single-metaheuristic baselines and, in many cases, even surpasses deterministic smoothing strategies, while substantially reducing overall evaluation cost. The approach offers a flexible, efficient tool for PNN learning with potential extensions to hyperparameter optimisation and cross-method population transfer.

Abstract

This study investigates the potential of hybrid metaheuristic algorithms to enhance the training of Probabilistic Neural Networks (PNNs) by leveraging the complementary strengths of multiple optimisation strategies. Traditional learning methods, such as gradient-based approaches, often struggle to optimise high-dimensional and uncertain environments, while single-method metaheuristics may fail to exploit the solution space fully. To address these challenges, we propose the constrained Hybrid Metaheuristic (cHM) algorithm, a novel approach that combines multiple population-based optimisation techniques into a unified framework. The proposed procedure operates in two phases: an initial probing phase evaluates multiple metaheuristics to identify the best-performing one based on the error rate, followed by a fitting phase where the selected metaheuristic refines the PNN to achieve optimal smoothing parameters. This iterative process ensures efficient exploration and convergence, enhancing the network's generalisation and classification accuracy. cHM integrates several popular metaheuristics, such as BAT, Simulated Annealing, Flower Pollination Algorithm, Bacterial Foraging Optimization, and Particle Swarm Optimisation as internal optimisers. To evaluate cHM performance, experiments were conducted on 16 datasets with varying characteristics, including binary and multiclass classification tasks, balanced and imbalanced class distributions, and diverse feature dimensions. The results demonstrate that cHM effectively combines the strengths of individual metaheuristics, leading to faster convergence and more robust learning. By optimising the smoothing parameters of PNNs, the proposed method enhances classification performance across diverse datasets, proving its application flexibility and efficiency.

Constrained Hybrid Metaheuristic Algorithm for Probabilistic Neural Networks Learning

TL;DR

This work tackles training Probabilistic Neural Networks (PNNs) in uncertain, high-dimensional settings by proposing a constrained Hybrid Metaheuristic (cHM) that dynamically blends multiple swarm-based optimisers. The method operates in two phases—probing to select the best performing metaheuristic by minimizing the error rate, then fitting to refine the smoothing parameters within the KDE framework—achieving faster convergence and improved generalisation. Empirical results on 16 diverse datasets show cHM often outperforms single-metaheuristic baselines and, in many cases, even surpasses deterministic smoothing strategies, while substantially reducing overall evaluation cost. The approach offers a flexible, efficient tool for PNN learning with potential extensions to hyperparameter optimisation and cross-method population transfer.

Abstract

This study investigates the potential of hybrid metaheuristic algorithms to enhance the training of Probabilistic Neural Networks (PNNs) by leveraging the complementary strengths of multiple optimisation strategies. Traditional learning methods, such as gradient-based approaches, often struggle to optimise high-dimensional and uncertain environments, while single-method metaheuristics may fail to exploit the solution space fully. To address these challenges, we propose the constrained Hybrid Metaheuristic (cHM) algorithm, a novel approach that combines multiple population-based optimisation techniques into a unified framework. The proposed procedure operates in two phases: an initial probing phase evaluates multiple metaheuristics to identify the best-performing one based on the error rate, followed by a fitting phase where the selected metaheuristic refines the PNN to achieve optimal smoothing parameters. This iterative process ensures efficient exploration and convergence, enhancing the network's generalisation and classification accuracy. cHM integrates several popular metaheuristics, such as BAT, Simulated Annealing, Flower Pollination Algorithm, Bacterial Foraging Optimization, and Particle Swarm Optimisation as internal optimisers. To evaluate cHM performance, experiments were conducted on 16 datasets with varying characteristics, including binary and multiclass classification tasks, balanced and imbalanced class distributions, and diverse feature dimensions. The results demonstrate that cHM effectively combines the strengths of individual metaheuristics, leading to faster convergence and more robust learning. By optimising the smoothing parameters of PNNs, the proposed method enhances classification performance across diverse datasets, proving its application flexibility and efficiency.
Paper Structure (17 sections, 10 equations, 4 figures, 11 tables, 1 algorithm)

This paper contains 17 sections, 10 equations, 4 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Motivation
  3. Contribution
  4. Paper organization
  5. Probabilistic neural network
  6. Structure of probabilistic neural network
  7. Training procedures
  8. The constrained Hybrid Metaheuristic (cHM) algorithm
  9. Proposed cHM algorithm
  10. Metaheuristic procedures
  11. Experimental results
  12. PNN training details
  13. Datasets
  14. Results of proposed learning procedure
  15. Metaheuristic training methods performance
  16. ...and 2 more sections

Figures (4)

  • Figure 1: Bar plot of single metaheuristic selection by the cHM algorithm for the Cancer dataset.
  • Figure 2: Dot plot of single metaheuristic selection by the cHM algorithm for the Cancer dataset. The count is presented for each iteration of the cHM method.
  • Figure 3: Bar plot of single metaheuristic selection by the cHM algorithm for the Vehicle dataset.
  • Figure 4: Dot plot of single metaheuristic selection by the cHM algorithm for the Vehicle dataset. The count is presented for each iteration of the cHM method.