Improved Replicable Boosting with Majority-of-Majorities

TL;DR

This work tackles the problem of achieving $\rho$-replicability in boosting with favorable data efficiency in the weak-to-strong learning setting. It introduces a two-layer approach: an improved inner replicable learner $rBoost^*$ and a replicable threshold subroutine $rThreshold$, orchestrated by a meta-boosting algorithm $rMetaBoost$ that uses rejection sampling to maintain dense reweightings. The main results establish $\rho$-replicability, $O(\ln(1/\varepsilon)/\gamma^2)$ inner calls to the weak learner, and a total sample complexity of $\tilde{O}\left( \frac{m_{\mathcal{W}(\Theta(\rho\gamma^2))}}{\varepsilon\gamma^2} + \frac{1}{\rho^2\varepsilon\gamma^3} \right)$, improving on prior bounds by reducing dependence on $\varepsilon$ and $\gamma$. A practical replicable threshold check is also introduced, expanding the toolkit for replicable algorithms and enabling more data-efficient replicable boosting in practice.

Abstract

We introduce a new replicable boosting algorithm which significantly improves the sample complexity compared to previous algorithms. The algorithm works by doing two layers of majority voting, using an improved version of the replicable boosting algorithm introduced by Impagliazzo et al. [2022] in the bottom layer.

Theorems & Definitions (20)

• Definition 1.1: Replicability impagliazzo2022reproducibility
• Theorem 1.2: rMetaBoost
• Lemma 1.3: rThreshold
• Theorem 1.4: rBoost$^*$
• Lemma 2.1: Rejection Sampling impagliazzo2022reproducibility
• Lemma 2.2: Composing Replicable Algorithms with Rejection Sampling impagliazzo2022reproducibility
• Lemma 2.3: Correctness
• proof
• Lemma 2.4: High density of $\mu_t$
• proof