Automatic Regularization for Linear MMSE Filters

TL;DR

The paper tackles the challenge of choosing the regularization parameter in linear MMSE filters. It adopts a Bayesian framework to infer α from data via marginal likelihood, enabling efficient fixed-point or Gull–MacKay iterations. The method applies to both Wiener-type error minimization and MVDR interference suppression, with demonstrations in system identification and beamforming showing near-oracle performance and robustness to changing conditions. Compared with Ledoit–Wolf and Hoerl–Kennard–Baldwin approaches, the proposed automatic regularization adapts to problem structure and data, offering a practical, data-driven solution for regularization in signal processing.

Abstract

In this work, we consider the problem of regularization in the design of minimum mean square error (MMSE) linear filters. Using the relationship with statistical machine learning methods, using a Bayesian approach, the regularization parameter is found from the observed signals in a simple and automatic manner. The proposed approach is illustrated in system identification and beamforming examples, where the automatic regularization is shown to yield near-optimal results.

Figures (7)

• Figure 1: Impulse response $\boldsymbol{h}$ generated using Werner2023.
• Figure 2: Values $\alpha^{(i)}$ obtained via fixed-point iteration \ref{['alpha.i+1.i.new']} (dashed-dotted lines) and via Gull-MacKay iteration \ref{['alpha.Gull.McKay']} (dashed lines) in different realizations of the data using $\alpha^{(0)}=0.5$; $N = 1000$, $M = 600$. Solid lines are constant, as they indicate an oracle-given $\hat{\alpha}$. Thick lines indicate averages over realizations shown with thin lines.
• Figure 3: Results obtained for different regularization methods, and SNR equal to (a,b) $0$dB and (c,d) $20$dB. In (a,c) we show the misalignment $\mathsf{m}(\alpha^{(I)})$\ref{['Misalignement.def']}, and $\hat{\mathsf{m}}$ given by \ref{['hat.M']}, while the corresponding values of $\alpha^{(I)}$ and $\hat{\alpha}$ are shown in (b,d). In (b) and (d), thick lines are averages of realizations shown with thin lines.
• Figure 4: Empirical frequency of violating condition \ref{['Condition.for.finite.alpha']} in the interference suppression example with 10000 independent realizations.
• Figure 5: Empirical frequency of the number of roots in the interference suppression example across different signals of interest for 10000 independent realizations of data, $M=10$ and (a) $N = 10000$, (b) $N = 10$.

Theorems & Definitions (2)

• Proposition 1: Roots of $L'(\alpha)$
• Proposition 2