Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Estimation Sample Complexity of a Class of Nonlinear Continuous-time Systems

Simon Kuang, Xinfan Lin

TL;DR

This work addresses the challenge of estimating parameters in a broad class of nonlinear continuous-time systems where the state includes derivatives and the dynamics are linear in the parameters. It introduces a differentiation-based approach that turns the problem into a linear regression with errors in both variables, combining an arbitrary-order finite-difference differentiator with ridge-regularized least squares. The authors prove a finite-sample mean absolute error bound $\mathbb{E}\|\hat{\theta}-\theta\| = O\left(n^{-\alpha/(2m+3)}\right)$ under Hölder regularity, and illustrate the method with numerical and theoretical examples, discussing bias-variance tradeoffs and practical robustness. The results provide the first nonasymptotic performance guarantees for this class of nonlinear continuous-time identification problems and offer a new perspective beyond traditional PEM or linear-system theory.

Abstract

We present a method of parameter estimation for large class of nonlinear systems, namely those in which the state consists of output derivatives and the flow is linear in the parameter. The method, which solves for the unknown parameter by directly inverting the dynamics using regularized linear regression, is based on new design and analysis ideas for differentiation filtering and regularized least squares. Combined in series, they yield a novel finite-sample bound on mean absolute error of estimation.

Estimation Sample Complexity of a Class of Nonlinear Continuous-time Systems

TL;DR

This work addresses the challenge of estimating parameters in a broad class of nonlinear continuous-time systems where the state includes derivatives and the dynamics are linear in the parameters. It introduces a differentiation-based approach that turns the problem into a linear regression with errors in both variables, combining an arbitrary-order finite-difference differentiator with ridge-regularized least squares. The authors prove a finite-sample mean absolute error bound under Hölder regularity, and illustrate the method with numerical and theoretical examples, discussing bias-variance tradeoffs and practical robustness. The results provide the first nonasymptotic performance guarantees for this class of nonlinear continuous-time identification problems and offer a new perspective beyond traditional PEM or linear-system theory.

Abstract

We present a method of parameter estimation for large class of nonlinear systems, namely those in which the state consists of output derivatives and the flow is linear in the parameter. The method, which solves for the unknown parameter by directly inverting the dynamics using regularized linear regression, is based on new design and analysis ideas for differentiation filtering and regularized least squares. Combined in series, they yield a novel finite-sample bound on mean absolute error of estimation.
Paper Structure (14 sections, 3 theorems, 14 equations, 4 figures, 3 tables)

This paper contains 14 sections, 3 theorems, 14 equations, 4 figures, 3 tables.

Table of Contents

  1. Related work
  2. Discretization does not solve Problem \ref{['problem']}
  3. Curve fitting does not solve Problem \ref{['problem']}
  4. Linear theory does not solve Problem \ref{['problem']}
  5. A differentiation-based approach
  6. Contributions
  7. Preliminaries
  8. Solution to Problem \ref{['problem']}
  9. Differentiation
  10. Regularized least squares
  11. Numerical example
  12. Theoretical example
  13. Conclusion
  14. Proof of Lemma \ref{['lemma:head-body-tail']}

Key Result

Theorem 6

Assume: Let $\hat{\theta}_n(\omega) \in \arg \max_{\theta} Q_n(\omega, \theta)$ be a measurable extremum. Then $\hat{\theta}_n \overset{\mathbb{P}}{\to} \theta_0$.

Figures (4)

  • Figure 1: Sampling distribution of $\hat{\theta}_1$ at different choices of $N$.
  • Figure 2: Sampling distribution of $\hat{\theta}_2$ at different choices of $N$.
  • Figure 3: Log-likelihood of $\theta_1$ at the true $\theta_2$.
  • Figure 4: Log-likelihood of $\theta_2$ at the true $\theta_1$.

Theorems & Definitions (3)

  • Theorem 6: amemiya_advanced_1985
  • Theorem 7
  • Theorem 8