Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Debiasing Continuous-time Nonlinear Autoregressions

Simon Kuang, Xinfan Lin

TL;DR

The paper addresses identifiability of continuous-time nonlinear autoregressive systems affine in parameters from noisy observations. It proposes direct-estimation methods that first differentiate the noisy time series using local polynomial regression and then apply plug-in linear estimation, but acknowledges bias from noise; to mitigate this, it develops two bias-correction strategies: bias-corrected least squares (BC) and an instrumental-variables (IV) estimator. The authors establish asymptotic consistency in a double limit $(n\to\infty, h\to 0)$ and demonstrate substantial finite-sample bias reduction on canonical nonlinear oscillators, including van der Pol and Lorenz systems. The results show BC and IV outperform plain LS in bias and quadratic risk across different noise levels, thereby broadening the class of continuous-time nonlinear autoregressions that can be consistently estimated by direct methods.

Abstract

We study how to identify a class of continuous-time nonlinear systems defined by an ordinary differential equation affine in the unknown parameter. We define a notion of asymptotic consistency as $(n, h) \to (\infty, 0)$, and we achieve it using a family of direct methods where the first step is differentiating a noisy time series and the second step is a plug-in linear estimator. The first step, differentiation, is a signal processing adaptation of the nonparametric statistical technique of local polynomial regression. The second step, generalized linear regression, can be consistent using a least squares estimator, but we demonstrate two novel bias corrections that improve the accuracy for finite $h$. These methods significantly broaden the class of continuous-time systems that can be consistently estimated by direct methods.

Debiasing Continuous-time Nonlinear Autoregressions

TL;DR

The paper addresses identifiability of continuous-time nonlinear autoregressive systems affine in parameters from noisy observations. It proposes direct-estimation methods that first differentiate the noisy time series using local polynomial regression and then apply plug-in linear estimation, but acknowledges bias from noise; to mitigate this, it develops two bias-correction strategies: bias-corrected least squares (BC) and an instrumental-variables (IV) estimator. The authors establish asymptotic consistency in a double limit and demonstrate substantial finite-sample bias reduction on canonical nonlinear oscillators, including van der Pol and Lorenz systems. The results show BC and IV outperform plain LS in bias and quadratic risk across different noise levels, thereby broadening the class of continuous-time nonlinear autoregressions that can be consistently estimated by direct methods.

Abstract

We study how to identify a class of continuous-time nonlinear systems defined by an ordinary differential equation affine in the unknown parameter. We define a notion of asymptotic consistency as , and we achieve it using a family of direct methods where the first step is differentiating a noisy time series and the second step is a plug-in linear estimator. The first step, differentiation, is a signal processing adaptation of the nonparametric statistical technique of local polynomial regression. The second step, generalized linear regression, can be consistent using a least squares estimator, but we demonstrate two novel bias corrections that improve the accuracy for finite . These methods significantly broaden the class of continuous-time systems that can be consistently estimated by direct methods.

Paper Structure

This paper contains 15 sections, 10 theorems, 41 equations, 4 figures, 5 tables.

Table of Contents

  1. Statistical primitives
  2. Ordinary least squares (OLS)
  3. Bias correction
  4. An instrumental variables method
  5. Filtering
  6. Numerical example: van der Pol oscillator
  7. Numerical example: Lorenz system
  8. Conclusion
  9. Related work
  10. Bias
  11. Process vs. output noise models
  12. Differentiation of noisy data
  13. Proof of Lemma \ref{['lem:consistent-sum']}
  14. Proof of Lemma \ref{['lem:consistent-sum-2nd']}
  15. Filtering details

Key Result

Lemma 6

Let $f$ be a smooth function with bounded first through third derivatives, and let $\{\hat{x}_j\}_{j \in [1\ldots n']}$ come from a $(\beta, \gamma)$-consistent filter. Then

Figures (4)

  • Figure 1: Kernel density estimate of the sampling distribution of the estimated $\theta = (\theta^1, \theta^2)$ of §\ref{['sec:example-vdp']} under different pairings of differentiation and regression methods. True values indicated by a vertical line.
  • Figure 2: Two numerically indistinguishable solutions fo the Lorenz system initial value problem.
  • Figure 3: Kernel density estimate of the sampling distribution of the estimated $A_0$ of §\ref{['sec:example-lorenz']} with $\sigma^2=0.1$. True values indicated by vertical lines.
  • Figure 4: Kernel density estimate of the sampling distribution of the estimated $A_0$ of §\ref{['sec:example-lorenz']} with $\sigma^2=10$. True values indicated by vertical lines.

Theorems & Definitions (17)

  • Definition 5
  • Lemma 6
  • Lemma 8
  • Remark 9
  • Lemma 10: Lipschitz continuity of matrix inversion
  • Theorem 11: LS Consistency
  • Theorem 12: BC Consistency
  • Remark 13
  • Remark 14
  • Theorem 15: IV Consistency
  • ...and 7 more