Randomized based restricted kernel machine for hyperspectral image classification
A. Quadir, M. Tanveer
TL;DR
This work introduces a novel randomized based restricted kernel machine ($R^2KM$) that blends the efficiency of RVFL networks with the nonlinear expressiveness of restricted kernel machines, facilitated by a layered, RBM-like energy formulation and Fenchel–Young conjugate duality. The model derives a closed-form-like update for hidden representations and a dual-bound objective, enabling scalable classification and regression in high-dimensional spaces. Generalization is theoretically grounded via empirical Rademacher complexity, and extensive experiments on hyperspectral imagery and UCI/KEEL benchmarks demonstrate consistent improvements over strong baselines, with statistical tests confirming significance. The approach offers a robust, scalable framework for complex data—especially HSIs—by effectively capturing spectral-spatial interactions while maintaining computational efficiency.
Abstract
In recent years, the random vector functional link (RVFL) network has gained significant popularity in hyperspectral image (HSI) classification due to its simplicity, speed, and strong generalization performance. However, despite these advantages, RVFL models face several limitations, particularly in handling non-linear relationships and complex data structures. The random initialization of input-to-hidden weights can lead to instability, and the model struggles with determining the optimal number of hidden nodes, affecting its performance on more challenging datasets. To address these issues, we propose a novel randomized based restricted kernel machine ($R^2KM$) model that combines the strehyperngths of RVFL and restricted kernel machines (RKM). $R^2KM$ introduces a layered structure that represents kernel methods using both visible and hidden variables, analogous to the energy function in restricted Boltzmann machines (RBM). This structure enables $R^2KM$ to capture complex data interactions and non-linear relationships more effectively, improving both interpretability and model robustness. A key contribution of $R^2KM$ is the introduction of a novel conjugate feature duality based on the Fenchel-Young inequality, which expresses the problem in terms of conjugate dual variables and provides an upper bound on the objective function. This duality enhances the model's flexibility and scalability, offering a more efficient and flexible solution for complex data analysis tasks. Extensive experiments on hyperspectral image datasets and real-world data from the UCI and KEEL repositories show that $R^2KM$ outperforms baseline models, demonstrating its effectiveness in classification and regression tasks.