Optimal Bayesian Affine Estimator and Active Learning for the Wiener Model

TL;DR

The paper develops a Bayesian estimation framework for Wiener models with known linear dynamics and an unknown static nonlinear block, deriving a closed-form optimal affine estimator expressed via dynamic basis statistics (DBS). It analyzes key properties including Bayesian unbiasedness, posterior updates, monotone error reduction, and consistency conditions; for Fourier bases, explicit DBS expressions reveal single-trajectory inconsistency under stochastic instability and motivate an active-learning input design to minimize estimation error. The approach is validated numerically, showing clear improvements over regularized least-squares methods, and the BAL variant with active learning yields the best performance in many scenarios. The work provides practical tools for accurate nonlinear system identification under correlated noise and presents an open-source MATLAB library to enable reproducibility and further research.

Abstract

This paper presents a Bayesian estimation framework for Wiener models, focusing on learning nonlinear output functions under known linear state dynamics. We derive a closed-form optimal affine estimator for the unknown parameters, characterized by the so-called "dynamic basis statistics" (DBS). Several features of the proposed estimator are studied, including Bayesian unbiasedness, closed-form posterior statistics, error monotonicity in trajectory length, and consistency condition (also known as persistent excitation). In the special case of Fourier basis functions, we demonstrate that the closed-form description is computationally available, as the Fourier DBS enjoys explicit expressions. Furthermore, we identify an inherent inconsistency in the Fourier bases for single-trajectory measurements, regardless of the input excitation. Leveraging the closed-form estimation error, we develop an active learning algorithm synthesizing input signals to minimize estimation error. Numerical experiments validate the efficacy of our approach, showing significant improvements over traditional regularized least-squares methods.

Figures (4)

• Figure 1: Squared errors of DLS, MLS, and BMS for different $\lambda$ values
• Figure 2: Distribution of squared error differences between pairs of estimators
• Figure 3: Squared errors of DLS, MLS, BMS, and BAL for varying $T$
• Figure 4: Distribution of squared errors for multiple $\tau$

Theorems & Definitions (18)

• Remark 2.1: Modelling setting
• Definition 1: Dynamic basis statistics
• Remark 3.1: MMSE estimate of elliptical distributions
• Theorem 3.2: Optimal Bayesian MMSE affine estimator
• Proposition 3.3: Bayesian unbiasedness
• Remark 3.4: Bayesian update via posterior distribution
• Corollary 3.5: Bayesian error monotonicity
• proof : Proof of Theorem \ref{['theorem:MMSE_LinearEstimator']}
• proof : Proof of Proposition \ref{['proposition:Unbiasedness']}
• proof : Proof of Corollary \ref{['corollary:Bayesian_Error_Reduction']}