Bayesian Modeling and Estimation of Linear Time-Varying Systems using Neural Networks and Gaussian Processes

Abstract

The identification of Linear Time-Varying (LTV) systems from input-output data is a fundamental yet challenging ill-posed inverse problem. This work introduces a unified Bayesian framework that models the system's impulse response, $h(t, τ)$, as a stochastic process. We decompose the response into a posterior mean and a random fluctuation term, a formulation that provides a principled approach for quantifying uncertainty, unifies intrinsic channel variability and epistemic uncertainty through a common posterior representation, and naturally defines a new, useful system class we term Linear Time-Invariant in Expectation (LTIE). To perform inference, we leverage modern machine learning techniques, including Bayesian neural networks and Gaussian Processes, using scalable variational inference. We demonstrate through a series of experiments that our framework can infer the properties of an LTI system from a single noisy input-output pair, including under deliberate additive-noise misspecification, achieve a lower overall error floor than the classical CCF stacking baseline in a simulated ambient noise tomography setting, and track a continuously varying LTV impulse response by using a structured Gaussian Process prior. This work provides a flexible and robust methodology for uncertainty-aware system identification in dynamic environments.

Figures (12)

• Figure B.1: The synthetic data used for experiment one- impulse response regression from a single observation.
• Figure B.2: Denoising performance of the Bayesian model in the single-observation experiment. The figure shows the posterior predictive distribution for the received signal, $\hat{g}[n]$ (a collection of green samples). By learning the impulse response, the model generates predictions that accurately reconstruct the clean signal while filtering the noise from the original observation.
• Figure B.3: Bayesian estimation of the impulse response, $h[k]$, from a single noisy observation. The model learns a full posterior distribution that successfully recovers the ground truth filter and quantifies the associated estimation uncertainty.
• Figure B.4: Propagation of uncertainty into the frequency domain. The variational posterior distribution over the impulse response induces a corresponding approximate posterior over the system's frequency and phase response, allowing for comprehensive uncertainty analysis.
• Figure B.5: Samples from the posterior distribution of the cross-correlation function (CCF) for the single-observation experiment. These plots show the estimated CCF distribution (green) robustly matching the ground truth (orange) while providing a credible interval, in contrast to the noisy observed CCF (red). Each panel shows a zoomed-in view of a different lag region.