Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gradient Boosted Filters For Signal Processing

Jose A. Lopez, Georg Stemmer, Hector A. Cordourier

TL;DR

This work addresses the limitation of traditional gradient boosted models in modeling dynamic signals by introducing gradient boosted filters (GBFs) that replace decision-tree weak learners with Hammerstein systems. The approach links GBFs to the Volterra/Wiener framework for nonlinear dynamic systems and explores separate and combined training approaches, leveraging closed-form Wiener-Hopf estimation to reduce parameter updates. Empirical results on a simple dynamical system and laptop acoustics demonstrate that GBFs can generalize beyond training data and achieve competitive, often superior, MSE performance compared to Wiener baselines and Volterra networks, while maintaining efficiency. The proposed method offers a compact, interpretable alternative for dynamic signal processing with potential extensions to multi-input/output settings and richer linear components.

Abstract

Gradient boosted decision trees have achieved remarkable success in several domains, particularly those that work with static tabular data. However, the application of gradient boosted models to signal processing is underexplored. In this work, we introduce gradient boosted filters for dynamic data, by employing Hammerstein systems in place of decision trees. We discuss the relationship of our approach to the Volterra series, providing the theoretical underpinning for its application. We demonstrate the effective generalizability of our approach with examples.

Gradient Boosted Filters For Signal Processing

TL;DR

This work addresses the limitation of traditional gradient boosted models in modeling dynamic signals by introducing gradient boosted filters (GBFs) that replace decision-tree weak learners with Hammerstein systems. The approach links GBFs to the Volterra/Wiener framework for nonlinear dynamic systems and explores separate and combined training approaches, leveraging closed-form Wiener-Hopf estimation to reduce parameter updates. Empirical results on a simple dynamical system and laptop acoustics demonstrate that GBFs can generalize beyond training data and achieve competitive, often superior, MSE performance compared to Wiener baselines and Volterra networks, while maintaining efficiency. The proposed method offers a compact, interpretable alternative for dynamic signal processing with potential extensions to multi-input/output settings and richer linear components.

Abstract

Gradient boosted decision trees have achieved remarkable success in several domains, particularly those that work with static tabular data. However, the application of gradient boosted models to signal processing is underexplored. In this work, we introduce gradient boosted filters for dynamic data, by employing Hammerstein systems in place of decision trees. We discuss the relationship of our approach to the Volterra series, providing the theoretical underpinning for its application. We demonstrate the effective generalizability of our approach with examples.
Paper Structure (9 sections, 8 equations, 12 figures, 1 table, 2 algorithms)

This paper contains 9 sections, 8 equations, 12 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Volterra Series
  3. Related Work
  4. Gradient Boosted Filters
  5. Experiments
  6. Example 1: A simple dynamical system
  7. Example 2: Laptop Acoustics
  8. Conclusions
  9. Additional Plots.

Figures (12)

  • Figure 1: A Hammerstein system with a polynomial non-linearity and FIR component.
  • Figure 2: Model output for input data with values between -1 and 1.
  • Figure 3: Model output for input data with values between -2 and 2.
  • Figure 4: Model output for input data with values between -4 and 4.
  • Figure 5: Reference and recorded (16 KHz) audio signals.
  • ...and 7 more figures

Theorems & Definitions (2)

  • Remark 3.1: Expressiveness.
  • Remark 3.2: Wiener-Hopf solution.