Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Derandomizing Simultaneous Confidence Regions for Band-Limited Functions by Improved Norm Bounds and Majority-Voting Schemes

Balázs Csanád Csáji, Bálint Horváth

TL;DR

This work improves nonparametric, nonasymptotic simultaneous confidence regions for band-limited functions within a Paley-Wiener RKHS. It tightens kernel-norm bounds through a uniformly-randomized Hoeffding bound and an empirical Bernstein bound, introducing a data-driven threshold to decide which bound to deploy. It also demonstrates that aggregating confidence sets from random subsamples via majority-voting techniques preserves simultaneous coverage while reducing variability and region size. Numerical experiments corroborate that the proposed refinements yield more stable, tighter confidence bands across inputs. Overall, the method advances distribution-free inference for band-limited regression with finite-sample guarantees and practical robustness.

Abstract

Band-limited functions are fundamental objects that are widely used in systems theory and signal processing. In this paper we refine a recent nonparametric, nonasymptotic method for constructing simultaneous confidence regions for band-limited functions from noisy input-output measurements, by working in a Paley-Wiener reproducing kernel Hilbert space. Kernel norm bounds are tightened using a uniformly-randomized Hoeffding's inequality for small samples and an empirical Bernstein bound for larger ones. We derive an approximate threshold, based on the sample size and how informative the inputs are, that governs which bound to deploy. Finally, we apply majority voting to aggregate confidence sets from random subsamples, boosting both stability and region size. We prove that even per-input aggregated intervals retain their simultaneous coverage guarantee. These refinements are also validated through numerical experiments.

Derandomizing Simultaneous Confidence Regions for Band-Limited Functions by Improved Norm Bounds and Majority-Voting Schemes

TL;DR

This work improves nonparametric, nonasymptotic simultaneous confidence regions for band-limited functions within a Paley-Wiener RKHS. It tightens kernel-norm bounds through a uniformly-randomized Hoeffding bound and an empirical Bernstein bound, introducing a data-driven threshold to decide which bound to deploy. It also demonstrates that aggregating confidence sets from random subsamples via majority-voting techniques preserves simultaneous coverage while reducing variability and region size. Numerical experiments corroborate that the proposed refinements yield more stable, tighter confidence bands across inputs. Overall, the method advances distribution-free inference for band-limited regression with finite-sample guarantees and practical robustness.

Abstract

Band-limited functions are fundamental objects that are widely used in systems theory and signal processing. In this paper we refine a recent nonparametric, nonasymptotic method for constructing simultaneous confidence regions for band-limited functions from noisy input-output measurements, by working in a Paley-Wiener reproducing kernel Hilbert space. Kernel norm bounds are tightened using a uniformly-randomized Hoeffding's inequality for small samples and an empirical Bernstein bound for larger ones. We derive an approximate threshold, based on the sample size and how informative the inputs are, that governs which bound to deploy. Finally, we apply majority voting to aggregate confidence sets from random subsamples, boosting both stability and region size. We prove that even per-input aggregated intervals retain their simultaneous coverage guarantee. These refinements are also validated through numerical experiments.

Paper Structure

This paper contains 8 sections, 6 theorems, 35 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Kernels and Band-Limited Functions
  3. Problem Setting
  4. Overview of the Original Construction
  5. Improving the Kernel Norm Bounds
  6. Merging Confidence Bands from Subsamples
  7. Numerical Experiments
  8. Conclusions

Key Result

Theorem 1

Let $X_1, \dots, X_n$ be i.i.d. $\sigma$-sub-Gaussian random variables and $U$ be a variable with uniform distribution on $(0,1)$, independent from $X_1, \dots, X_n$. Then, for any $\alpha \in (0,1)$, where $\bar{X}=(1/n)\sum_{i=1}^nX_i$ denotes the sample average.

Figures (3)

  • Figure 1: Differences of the upper bounds $(\alpha=0.1)$ and the original kernel norm of the regression function for $n=50$ and $n=500$.
  • Figure 2: $90\,\%$ combined confidence bands using random ordering, \ref{['set-permutation']}.
  • Figure 3: $90\,\%$ combined conf. bands using random thresholding, \ref{['set-thresholding-1']}.

Theorems & Definitions (11)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Theorem 1: Uniformly-Randomized Hoeffding's Inequality
  • Definition 5
  • Theorem 2: Empirical Bernstein’s Inequality
  • Theorem 3
  • Theorem 4
  • Theorem 5
  • ...and 1 more