Automatic Construction of Pattern Classifiers Capable of Continuous Incremental Learning and Unlearning Tasks Based on Compact-Sized Probabilistic Neural Network

TL;DR

This work tackles the problem of building pattern classifiers that can continually learn and unlearn without manual hyperparameter tuning. It introduces a compact-sized probabilistic neural network (CS-PNN) with a dynamic per-RBF radius and a data-driven construction/reconstruction process, enabling continuous incremental learning and unlearning (IIL/CIL and CDL) without iterative hyperparameter optimization. The approach demonstrates that CS-PNN can achieve competitive accuracy with far fewer hidden units than a standard PNN and often outperform replay-based DNNs in CIL scenarios, while remaining adaptable to continuous unlearning tasks. The findings suggest practical, rapid, and flexible pattern recognition suitable for real-time continual learning applications, with future work targeting larger datasets and parallelized implementations.

Abstract

This paper proposes a novel approach to pattern classification using a probabilistic neural network model. The strategy is based on a compact-sized probabilistic neural network capable of continuous incremental learning and unlearning tasks. The network is constructed/reconstructed using a simple, one-pass network-growing algorithm with no hyperparameter tuning. Then, given the training dataset, its structure and parameters are automatically determined and can be dynamically varied in continual incremental and decremental learning situations. The algorithm proposed in this work involves no iterative or arduous matrix-based parameter approximations but a simple data-driven updating scheme. Simulation results using nine publicly available databases demonstrate the effectiveness of this approach, showing that compact-sized probabilistic neural networks constructed have a much smaller number of hidden units compared to the original probabilistic neural network model and yet can achieve a similar classification performance to that of multilayer perceptron neural networks in standard classification tasks, while also exhibiting sufficient capability in continuous class incremental learning and unlearning tasks.

Figures (5)

• Figure 1: A PNN (left) and its topologically equivalent structure with $N_c$ subnets (right) Hoya-1998. In the figure, the matrices $\bm{C}=[\bm{c}_1, \bm{c}_2, \ldots, \bm{c}_{N_h}]$ and $\bm{W}=\{w_{jk}\}$ where $w_{jk}=0/1$; all the input units $x_i$$(i=1, 2, \ldots, N_d)$ are connected to each hidden layer unit $h_j$$(j=1, 2, \ldots, N_h)$ with the respective weight values denoted by the elements in $\bm{c}_j$. Each of $h_j$ is connected only to one of the output units $o_k$$(k=1, 2, \ldots, N_c)$ with the weight values $w_{jk}=1$ (if it belongs to the same class as $o_k$) and $0$ (otherwise).
• Figure 2: Transition of the classification accuracy (averaged) for the CIL tasks using the seven datasets: a) isolet, b) letter-recognition, c) MNIST, d) optdigits, e) pendigits, f) sat, and g) segmentation.
• Figure 3: Transition of the number of RBFs (averaged) generated within the CS-PNN for the CIL tasks using the seven datasets: a) isolet, b) letter-recognition, c) MNIST, d) optdigits, e) pendigits, f) sat, and g) segmentation.
• Figure 4: Transition of the classification accuracy (averaged) for the CUIL tasks using the seven datasets: a) isolet, b) letter-recognition, c) MNIST, d) optdigits, e) pendigits, f) sat, and g) segmentation.
• Figure 5: Transition of the number of RBFs (averaged) generated within the CS-PNN for the CUIL tasks using the seven datasets: a) isolet, b) letter-recognition, c) MNIST, d) optdigits, e) pendigits, f) sat, and g) segmentation.