Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Kernel Stochastic Configuration Networks for Nonlinear Regression

Yongxuan Chen, Dianhui Wang

TL;DR

Experimental results with comparisons against existing solutions clearly demonstrate that the proposed KSCN remarkably outperforms the original SCNs and some typical kernel methods for resolving nonlinear regression problems in terms of the learning performance, the model's stability and robustness with respect to the kernel parameter settings.

Abstract

Stochastic configuration networks (SCNs), as a class of randomized learner models, are featured by its way of random parameters assignment in the light of a supervisory mechanism, resulting in the universal approximation property at algorithmic level. This paper presents a kernel version of SCNs, termed KSCNs, aiming to enhance model's representation learning capability and performance stability. The random bases of a built SCN model can be used to span a reproducing kernel Hilbert space (RKHS), followed by our proposed algorithm for constructing KSCNs. It is shown that the data distribution in the reconstructive space is favorable for regression solving and the proposed KSCN learner models hold the universal approximation property. Three benchmark datasets including two industrial datasets are used in this study for performance evaluation. Experimental results with comparisons against existing solutions clearly demonstrate that the proposed KSCN remarkably outperforms the original SCNs and some typical kernel methods for resolving nonlinear regression problems in terms of the learning performance, the model's stability and robustness with respect to the kernel parameter settings.

Kernel Stochastic Configuration Networks for Nonlinear Regression

TL;DR

Experimental results with comparisons against existing solutions clearly demonstrate that the proposed KSCN remarkably outperforms the original SCNs and some typical kernel methods for resolving nonlinear regression problems in terms of the learning performance, the model's stability and robustness with respect to the kernel parameter settings.

Abstract

Stochastic configuration networks (SCNs), as a class of randomized learner models, are featured by its way of random parameters assignment in the light of a supervisory mechanism, resulting in the universal approximation property at algorithmic level. This paper presents a kernel version of SCNs, termed KSCNs, aiming to enhance model's representation learning capability and performance stability. The random bases of a built SCN model can be used to span a reproducing kernel Hilbert space (RKHS), followed by our proposed algorithm for constructing KSCNs. It is shown that the data distribution in the reconstructive space is favorable for regression solving and the proposed KSCN learner models hold the universal approximation property. Three benchmark datasets including two industrial datasets are used in this study for performance evaluation. Experimental results with comparisons against existing solutions clearly demonstrate that the proposed KSCN remarkably outperforms the original SCNs and some typical kernel methods for resolving nonlinear regression problems in terms of the learning performance, the model's stability and robustness with respect to the kernel parameter settings.

Paper Structure

This paper contains 16 sections, 21 equations, 15 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Stochastic Configuration Networks
  4. Reproducing Kernel Hilbert Space
  5. Kernel Stochastic Configuration Networks
  6. Construction of KSCNs
  7. Convergence Analysis
  8. Training Strategy of KSCNs
  9. Algorithm Description
  10. Performance Analysis
  11. Performance Evaluation
  12. Numerical Example
  13. Debutanizer Column Application
  14. Power Load Forecasting
  15. Robustness Analysis to Parameter Settings
  16. ...and 1 more sections

Figures (15)

  • Figure 1: An illustration of deducting nonlinearities by high-dimensional mapping: (a) the function plot of $f(x)=2x+xsin(x)$ where $x\in[0,10]$. (b) the function plot of $f[\phi(x)]=[2\; 1]\cdot\phi(x)$ where $\phi(x)=[x\; xsin(x)]^T$.
  • Figure 2: The schematic of SCNs.
  • Figure 3: The schematic of KSCNs.
  • Figure 4: An illustration about how different distributions of Gram matrices’ eigenvalues affect the regression performances.
  • Figure 5: The distributions of eigenvalues of different kernel Gram matrices.
  • ...and 10 more figures