Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Orthogonal Floating Search Algorithms: From The Perspective of Nonlinear System Identification

Faizal Hafiz, Akshya Swain, Eduardo Mendes

TL;DR

This paper addresses nonlinear system identification via structure selection for polynomial NARX models. It introduces the Orthogonal Floating Search Framework, which embeds orthogonalization into floating search algorithms (OSF, OIF, O2S) to evaluate term significance using ERR and to backtrack for improved subsets, thereby mitigating the nesting effect of traditional OFR-ERR. The framework demonstrates robust identification across benchmark nonlinear systems, including scenarios with slow-varying input and discretization of continuous-time models, and shows that model-order selection via information criteria yields accurate knee-points. The findings indicate that simple ERR metrics can be highly effective when coupled with flexible, orthogonal floating search strategies, offering practical and scalable structure identification for nonlinear system identification.

Abstract

The present study proposes a new Orthogonal Floating Search framework for structure selection of nonlinear systems by adapting the existing floating search algorithms for feature selection. The proposed framework integrates the concept of orthogonal space and consequent Error-Reduction-Ratio (ERR) metric with the existing floating search algorithms. On the basis of this framework, three well-known feature selection algorithms have been adapted which include the classical Sequential Forward Floating Search (SFFS), Improved sequential Forward Floating Search (IFFS) and Oscillating Search (OS). This framework retains the simplicity of classical Orthogonal Forward Regression with ERR (OFR-ERR) and eliminates the nesting effect associated with OFR-ERR. The performance of the proposed framework has been demonstrated considering several benchmark non-linear systems. The results show that most of the existing feature selection methods can easily be tailored to identify the correct system structure of nonlinear systems.

Orthogonal Floating Search Algorithms: From The Perspective of Nonlinear System Identification

TL;DR

This paper addresses nonlinear system identification via structure selection for polynomial NARX models. It introduces the Orthogonal Floating Search Framework, which embeds orthogonalization into floating search algorithms (OSF, OIF, O2S) to evaluate term significance using ERR and to backtrack for improved subsets, thereby mitigating the nesting effect of traditional OFR-ERR. The framework demonstrates robust identification across benchmark nonlinear systems, including scenarios with slow-varying input and discretization of continuous-time models, and shows that model-order selection via information criteria yields accurate knee-points. The findings indicate that simple ERR metrics can be highly effective when coupled with flexible, orthogonal floating search strategies, offering practical and scalable structure identification for nonlinear system identification.

Abstract

The present study proposes a new Orthogonal Floating Search framework for structure selection of nonlinear systems by adapting the existing floating search algorithms for feature selection. The proposed framework integrates the concept of orthogonal space and consequent Error-Reduction-Ratio (ERR) metric with the existing floating search algorithms. On the basis of this framework, three well-known feature selection algorithms have been adapted which include the classical Sequential Forward Floating Search (SFFS), Improved sequential Forward Floating Search (IFFS) and Oscillating Search (OS). This framework retains the simplicity of classical Orthogonal Forward Regression with ERR (OFR-ERR) and eliminates the nesting effect associated with OFR-ERR. The performance of the proposed framework has been demonstrated considering several benchmark non-linear systems. The results show that most of the existing feature selection methods can easily be tailored to identify the correct system structure of nonlinear systems.

Paper Structure

This paper contains 20 sections, 19 equations, 10 figures, 8 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. The Polynomial NARX Model
  4. Orthogonal Regression
  5. Proposed Orthogonal Floating Search Framework
  6. Term Significance
  7. Orthogonal Sequential Floating Search
  8. Orthogonal Improved Floating Search
  9. Orthogonal Oscillating Search
  10. Selection of Subset Cardinality (Model Order Selection)
  11. Investigation Framework
  12. Test Non-linear Systems
  13. Search Outcomes
  14. Results
  15. Comparative Evaluation
  16. ...and 5 more sections

Figures (10)

  • Figure 1: Search Behavior of OSF for the numerical example with $\xi=6$. Note that the 'backtracking' at Step 6 led to better subsets, e.g., $J(\mathcal{X}_2^+) > J(\mathcal{X}_2)$, $J(\mathcal{X}_3^+) > J(\mathcal{X}_3)$ and so on.
  • Figure 2: Search Behavior of OIF for the numerical example with $\xi=5$. Note the backtracking after $\mathcal{X}_4$ leading to better subsets, $\mathcal{X}_2^+$, $\mathcal{X}_3^+$ and $\mathcal{X}_4^+$.
  • Figure 3: The variation in subset cardinality during the O$^2$S Search
  • Figure 4: The selection of subset cardinality or 'model order'. $\xi^{\star}$ denotes the known cardinality of the system. Note that the knee-point for all criteria is obtained at $\xi=5$.
  • Figure 5: Model Order Selection. '$\xi^{\star}$' denotes the known cardinality of the system.
  • ...and 5 more figures

Theorems & Definitions (4)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4