On the Coordinate Change to the First-Order Spline Kernel for Regularized Impulse Response Estimation

TL;DR

This work generalizes coordinate-change kernels for regularized impulse-response estimation by using the impulse response $g_0(t)$ of a stable, strictly proper transfer function $G_0(s)$ to define $K_{G_0}(\\tau_1,\\tau_2)=\\min(|g_0(\\tau_1)|,|g_0(\\tau_2)|)$, extending the classic exponential-change kernels that yield TC, SS, and DC. It proves positivity, stability, zero-crossing inheritance, and a maximum-entropy interpretation, and provides a spectral decomposition for the multiple-pole spline kernel $G_0(s)=\\kappa/(s+\\alpha)^{n+1}$, enabling finite-dimensional RKHS representations via eigenfunctions $\\phi_{n,i}$ and eigenvalues $\\lambda_{n,i}$. An illustrative example with a two-pole target demonstrates that the proposed kernel, tuned via empirical Bayes, can match or exceed TC performance and better suppress overfitting in certain settings. The results offer a principled framework for designing kernel-based priors that encode specific a priori system dynamics through coordinate changes, with practical implications for robust impulse-response estimation.

Abstract

The so-called tuned-correlated kernel (sometimes also called the first-order stable spline kernel) is one of the most widely used kernels for the regularized impulse response estimation. This kernel can be derived by applying an exponential decay function as a coordinate change to the first-order spline kernel. This paper focuses on this coordinate change and derives new kernels by investigating other coordinate changes induced by stable and strictly proper transfer functions. It is shown that the corresponding kernels inherit properties from these coordinate changes and the first-order spline kernel. In particular, they have the maximum entropy property and moreover, the inverse of their Gram matrices has sparse structure. In addition, the spectral analysis of some special kernels are provided. Finally, a numerical example is given to show the efficacy of the proposed kernel.

Figures (12)

• Figure 1: Sparsity pattern of matrices
• Figure 4: Illustration of $m(\tau)$ with $n=1, \alpha=1$
• Figure 5: Illustration of $\frac{dm}{d\tau}$ with $n=1, \alpha=1$
• Figure 6: Illustration of $\phi_{n,i}(\tau)$ with $n=1,\alpha=1$
• Figure 7: Illustration of $\|\bm{K}-\bm{K}_M\|_{\rm FRO}$

Theorems & Definitions (2)

• Example 1
• Example 2