Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multiparameter regularization and aggregation in the context of polynomial functional regression

Elke R. Gizewski, Markus Holzleitner, Lukas Mayer-Suess, Sergiy Pereverzyev, Sergei V. Pereverzyev

TL;DR

This study introduces an algorithm for multiple parameter regularization and presents a theoretically grounded method for dealing with the associated parameters, which facilitates the aggregation of models with varying regularization parameters.

Abstract

Most of the recent results in polynomial functional regression have been focused on an in-depth exploration of single-parameter regularization schemes. In contrast, in this study we go beyond that framework by introducing an algorithm for multiple parameter regularization and presenting a theoretically grounded method for dealing with the associated parameters. This method facilitates the aggregation of models with varying regularization parameters. The efficacy of the proposed approach is assessed through evaluations on both synthetic and some real-world medical data, revealing promising results.

Multiparameter regularization and aggregation in the context of polynomial functional regression

TL;DR

This study introduces an algorithm for multiple parameter regularization and presents a theoretically grounded method for dealing with the associated parameters, which facilitates the aggregation of models with varying regularization parameters.

Abstract

Most of the recent results in polynomial functional regression have been focused on an in-depth exploration of single-parameter regularization schemes. In contrast, in this study we go beyond that framework by introducing an algorithm for multiple parameter regularization and presenting a theoretically grounded method for dealing with the associated parameters. This method facilitates the aggregation of models with varying regularization parameters. The efficacy of the proposed approach is assessed through evaluations on both synthetic and some real-world medical data, revealing promising results.
Paper Structure (14 sections, 8 theorems, 78 equations, 2 figures, 2 tables)

This paper contains 14 sections, 8 theorems, 78 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Setting
  3. Overall setting and assumptions
  4. Operator norms and related auxiliary estimates
  5. Multiparameter regularization
  6. Aggregation of multiple regularized polynomial functional models
  7. Bound on $\|G-\tilde{G}\|_{\mathcal{L}(\mathbb{R}^R)}$:
  8. Bound on $\|\tilde{g}-\bar{g}\|_{\mathbb{R}^{R}}$:
  9. Bound on $\|G^{-1}\|_{\mathcal{L}(\mathbb{R}^R)}$:
  10. Bound on $\|\tilde{c}\|_{\mathbb{R}^{R}}$:
  11. Experimental evaluation
  12. Toy example
  13. Stenosis Data
  14. Acknowledgements

Key Result

Lemma 2.1

Let $\text{HS}(\mathcal{H}_1, \mathcal{H}_2)$ denote the Hilbert space of Hilbert-Schmidt operators between Hilbert spaces $\mathcal{H}_1$ and $\mathcal{H}_2$. For simplicity let us also use $\text{HS}(\mathcal{H}_1, \mathcal{H}_1)=\text{HS}(\mathcal{H}_1).$ Under Assumption ass:unif we have that

Figures (2)

  • Figure 1: Error curves for all possible choices of $\bm{\lambda}$. Red line depicts error rate of $3.14$
  • Figure 2: Error curve for aggregation. Red line depicts error rate of $3.14$

Theorems & Definitions (11)

  • Remark 2.1
  • Lemma 2.1: Lemma 1 in holzleitner2023regularized
  • Lemma 2.2: Lemma 2 in holzleitner2023regularized
  • Lemma 2.3: compare with Lemma 4 in holzleitner2023regularized
  • Lemma 2.4: see e.g. Theorem 3.3.4. in yurinsky1995sums
  • Lemma 3.1
  • Theorem 4.1
  • Lemma 4.1
  • Remark 4.1
  • Remark 4.2
  • ...and 1 more