Generalization Bounds and Stopping Rules for Learning with Self-Selected Data

TL;DR

This work develops universal generalization guarantees for reciprocal learning, a broad class of feedback-driven data-selection methods that iteratively modify the training sample based on the model. By embedding reciprocal learning into multi-shot ERM and reinterpreting the induced distributional shifts through Wasserstein ambiguity sets and covering entropy, the authors derive bounds on the generalization gap and excess risk that require only verifiable algorithmic conditions and no distributional assumptions on self-selected data. They provide both convergent-solution bounds and anytime bounds that hold uniformly across iterations, enabling stopping rules that guarantee out-of-sample performance. The SSL setting is used to illustrate stopping rules in a practical, safety-critical context, and the framework offers a path toward designing new reciprocal algorithms with principled generalization guarantees. Overall, the paper bridges theory and practice for learning from self-selected data across active learning, SSL, bandits, boosting, and beyond.

Abstract

Many learning paradigms self-select training data in light of previously learned parameters. Examples include active learning, semi-supervised learning, bandits, or boosting. Rodemann et al. (2024) unify them under the framework of "reciprocal learning". In this article, we address the question of how well these methods can generalize from their self-selected samples. In particular, we prove universal generalization bounds for reciprocal learning using covering numbers and Wasserstein ambiguity sets. Our results require no assumptions on the distribution of self-selected data, only verifiable conditions on the algorithms. We prove results for both convergent and finite iteration solutions. The latter are anytime valid, thereby giving rise to stopping rules for a practitioner seeking to guarantee the out-of-sample performance of their reciprocal learning algorithm. Finally, we illustrate our bounds and stopping rules for reciprocal learning's special case of semi-supervised learning.

Figures (4)

• Figure 1: Left: Bound on Wasserstein-$p$ distance $W_p$ between law $\mathbb P$ and initial i.i.d. sample $\hat{\mathbb P}_0$ : $W_{p}(\mathbb P,\hat{\mathbb P}_0) \leq \beta_0$fournier2015rate. Right: Reciprocal distortion bound between initial i.i.d. sample $\hat{\mathbb P}_0$ and sample $\hat{\mathbb P}_T$ in reciprocal learning iteration $T$: $W_p(\hat{\mathbb P}_0, \hat{\mathbb P}_{T}) \leq \beta_T$ (Lemma \ref{['lemma:recipr-distortion']}).
• Figure 2: Left: Illustration of two possible samples $\hat{\mathbb P}_T$ to end up with in reciprocal learning. Right: Wasserstein ball on the sample distortion in reciprocal learning. It results from bounding the Wasserstein distance between the sample $\hat{\mathbb P}_{T}$ in $T$ and the law $\mathbb P$ by $\beta_0 + \beta_T$ (as illustrated in Figure \ref{['fig:illu-wasserstein-space']}) via the triangle inequality.
• Figure 3: A simple example of reciprocal learning: Self-training for binary classification. Features $x$ are chosen based on current model $\theta$, then added to the sample together with self-predicted (pseudo-)labels $\hat{y}(\theta,x)$. Replicated from goschenhofer2023reducing.
• Figure 4: Illustration of the generalization gap bound from Equation \ref{['eq:bound-example']} with $\delta \in \{0.05,0.1\}$, i.e., the bounds hold with probability of at least $0.95$ and $0.9$, respectively. Figure shows total bound (solid blue) and its components: the initial gap (solid red) and the reciprocal gap (dotted green).

Theorems & Definitions (18)

• Definition 1: Covering Entropy Integral
• Definition 2: Learner
• Definition 3: Risk and Loss
• Definition 4: One-Shot ERM
• Definition 5: Reciprocal Learning and Sample Adaptation
• Definition 6: Wasserstein-$p$ distance
• Lemma 7: Kantorovich-Rubinstein Lemma
• Theorem 8: Convergence of Reciprocal Learning
• Definition 9: Ambiguity set
• Lemma 10: Reciprocal Distortion Bound