Influence functions and regularity tangents for efficient active learning
Frederik Eaton
TL;DR
This work introduces a curvature-aware active-learning approach for regression by computing a regularity tangent $\frac{\mathrm{d}\theta^*}{\mathrm{d}s}$ alongside model training and using the inner product with the loss gradient as a data-point curiosity score. By linking the score to model complexity through the regularizer $R(s,\theta)$ and leveraging influence-function theory, the authors derive efficient, scalable query heuristics for selecting informative unlabeled points, including unlabeled variants and squared-influence measures. They formalize the regularity-tangent calculus, present a polynomial-regression example with LOOCV-guided regularity, and explore multi-user and hierarchical-regularization extensions, showing how global parameters and per-user deviations interact through $H^{-1}$-based adjoints. Finally, they propose SGDF and LiSSA-based methods to compute Hessian-vector products efficiently in large models, enabling practical deployment of curvature-informed active learning with modest storage overhead.
Abstract
In this paper we describe an efficient method for providing a regression model with a sense of curiosity about its data. In the field of machine learning, our framework for representing curiosity is called Active Learning, which concerns the problem of automatically choosing data points for which to query labels in the semi-supervised setting. The methods we propose are based on computing a "regularity tangent" vector that can be calculated (with only a constant slow-down) together with the model's parameter vector during training. We then take the inner product of this tangent vector with the gradient vector of the model's loss at a given data point to obtain a measure of the influence of that point on the complexity of the model. In the simplest instantiation, there is only a single regularity tangent vector, of the same dimension as the parameter vector. Thus, in the proposed technique, once training is complete, evaluating our "curiosity" about a potential query data point can be done as quickly as calculating the model's loss gradient at that point. The new vector only doubles the amount of storage required by the model. We show that the quantity computed by our technique is an example of an "influence function", and that it measures the expected squared change in model complexity incurred by up-weighting a given data point. We propose a number of ways for using this and other related quantities to choose new training data points for a regression model.