Reciprocal Learning

TL;DR

It is found that reciprocal learning algorithms converge at linear rates to an approximately optimal model under relatively mild assumptions on the loss function, if their predictions are probabilistic and the sample adaption is both non-greedy and either randomized or regularized.

Abstract

We demonstrate that a wide array of machine learning algorithms are specific instances of one single paradigm: reciprocal learning. These instances range from active learning over multi-armed bandits to self-training. We show that all these algorithms do not only learn parameters from data but also vice versa: They iteratively alter training data in a way that depends on the current model fit. We introduce reciprocal learning as a generalization of these algorithms using the language of decision theory. This allows us to study under what conditions they converge. The key is to guarantee that reciprocal learning contracts such that the Banach fixed-point theorem applies. In this way, we find that reciprocal learning algorithms converge at linear rates to an approximately optimal model under relatively mild assumptions on the loss function, if their predictions are probabilistic and the sample adaption is both non-greedy and either randomized or regularized. We interpret these findings and provide corollaries that relate them to specific active learning, self-training, and bandit algorithms.

Figures (5)

• Figure 1: (A) Classical machine learning fits a model from the model space (restricted by red curve) to a realized sample from the sample space (blue-grey); figure replicated from The Elements of Statistical Learninghastie2009elements. (B) In reciprocal learning, the realized sample is no longer static, but changes in response to the fit. Grey ellipse indicates restriction of sample space in $t=2$ through realization in $t=1$. Sample in $t$ thus depends on model in $t-1$and sample in $t-1$.
• Figure 2: Data regularization is symmetrical to classical regularization, see illustration in The Elements of Statistical Learninghastie2009elements.
• Figure 3: Reciprocal learning converges if the change in the data (purple) is bounded by the change in the model (yellow).
• Figure 4: (A) Classical one-shot machine learning fits a model from the model space (restricted by red curve) to a realized sample from the sample space (blue-grey), see hastie2009elements. (B) Reciprocal learning fits a model from the model space (restricted by red curve) to a realized sample from the sample space (blue-grey) that depends on the previous model fit, see Figure . (C) In online learning, there is no interaction between sample in $t$ and model in $t-1$.
• Figure 5: (A) Reciprocal learning fits a model from the model space (restricted by red curve) to a realized sample from the sample space (blue-grey) that depends on the previous model fit, see Figure . (B) In performative prediction, the population, not the sample, changes in response to the model fit. In other words, reciprocal learning algorithms have a static inference goal, while performative prediction is concerned with moving targets.

Theorems & Definitions (32)

• Definition 1: Reciprocal Learning, informal
• Definition 2: Data Selection
• Definition 3: Data Regularization
• Definition 4: Data Removal Function
• Definition 5: Sample Adaption
• Definition 6: Greedy Reciprocal Learning
• Definition 7: Non-Greedy Reciprocal Learning
• Definition 8: Convergence of Reciprocal Learning
• Definition 9: Optimal Data-Parameter Combination
• Example 1: Self-Training