On the impact of measure pre-conditionings on general parametric ML models and transfer learning via domain adaptation

TL;DR

This work introduces measure pre-conditioning as a principled approach to modify training data distributions in order to improve convergence and computational efficiency of general parametric ML models, while preserving the limiting problem. It leverages Gamma-convergence and a full learner recovery framework to guarantee that minimizers under pre-conditioned measures converge to the true optimizer, enabling stable domain adaptation and transfer learning. The paper surveys a spectrum of non-parametric, measure-based pre-conditioners (e.g., Wasserstein barycenters, entropic and class-based barycenters, MMD-regularized measures) and connects them to empirical density estimation, OT-based domain adaptation, and WGANs, with theoretical results and numerical demonstrations on tasks like MNIST with Gaussian blur and conditional average-guess adaptation. The framework provides practical recipes for selecting measures, discusses convergence properties under various topologies, and outlines directions for extending measure pre-conditioning to broader ML settings and transfer-learning scenarios.

Abstract

We study a new technique for understanding convergence of learning agents under small modifications of data. We show that such convergence can be understood via an analogue of Fatou's lemma which yields gamma-convergence. We show it's relevance and applications in general machine learning tasks and domain adaptation transfer learning.

Figures (11)

• Figure 1: Unblurred, $\sigma = 0$
• Figure 2: Blurred, $\sigma = 0.5$
• Figure 3: Blurred, $\sigma = 1$
• Figure 4: Loss function during training
• Figure 5: Accuracy of the model during training

Theorems & Definitions (55)

• Proposition 1
• Theorem 2
• Definition 1
• Remark 1
• Theorem 3
• Theorem 4
• Definition 2
• Remark 2
• Theorem 5
• proof