Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation

TL;DR

The paper tackles robustness gaps in meta-learning caused by distribution shifts by explicitly modeling task distributions over task identifiers and optimizing robust fast adaptation as a Stackelberg game. Task distributions are generated adversarially via normalizing flows, with a KL-based constraint ensuring shifts remain within a tolerable region, and the meta-learner acts as the leader while the distribution adversary acts as the follower. The authors establish convergence to a local Stackelberg equilibrium and derive a generalization bound under distribution shifts, while demonstrating improved robustness (via CVaR) on sinusoid, Acrobot, Pendulum, and meta-reinforcement learning benchmarks. The approach also reveals interpretable task-structure patterns in the learned distributions and provides code for replication, representing a practical advance for robust fast adaptation in real-world, shift-prone settings.

Abstract

Meta-learning is a practical learning paradigm to transfer skills across tasks from a few examples. Nevertheless, the existence of task distribution shifts tends to weaken meta-learners' generalization capability, particularly when the training task distribution is naively hand-crafted or based on simple priors that fail to cover critical scenarios sufficiently. Here, we consider explicitly generative modeling task distributions placed over task identifiers and propose robustifying fast adaptation from adversarial training. Our approach, which can be interpreted as a model of a Stackelberg game, not only uncovers the task structure during problem-solving from an explicit generative model but also theoretically increases the adaptation robustness in worst cases. This work has practical implications, particularly in dealing with task distribution shifts in meta-learning, and contributes to theoretical insights in the field. Our method demonstrates its robustness in the presence of task subpopulation shifts and improved performance over SOTA baselines in extensive experiments. The code is available at the project site https://sites.google.com/view/ar-metalearn.

Figures (18)

• Figure 1: Diagram of Generating Task Distribution as the Adversary in Meta-Learning. Here, the initial task distribution $p_{0}(\tau)$ is a uniform distribution governed by two task identifiers $[\nu,\mu]$. Then, it is transformed into an explicit distribution $p_{\bm\phi}(\tau)$ with the help of normalizing flows $\texttt{NF}_{\bm\phi}$.
• Figure 2: Diagram of Adversarially Task Robust Meta Learning. The proposed framework consists of two players, the distribution adversary and the meta player, in the game of meta-learning. On the left side of the figure: the distribution adversary seeks to transform the distribution from an initial task distribution, e.g., $\mathcal{N}(0,I_d)$ or $\mathcal{U}[a,b]$, via the neural network parameterized by $\bm\phi$ with the purpose of deteriorating meta player's fast adaptation performance. On the right side of the figure: the meta player parameterized by $\bm\theta$ attempts to learn robust strategies for fast adaptation in sampled worst-case tasks (MAML algorithm finn2017model as an illustration).
• Figure 3: Some Benchmarks in Evaluation. Blue-marked variables in the illustration denote task identifiers that guide the configuration of a specific task. We place distributions over these task identifiers in generating diverse tasks for meta-learning.
• Figure 4: Meta Testing Returns in Point Robot Navigation Tasks (4 runs). The charts report average and $\text{CVaR}_{\alpha}$ returns with $\alpha=0.5$ in initial and adversarial distributions, with standard error bars indicated by black vertical lines. The higher, the better.
• Figure 5: Meta Testing Returns in Ant Pos Tasks (4 runs). The charts report average and $\text{CVaR}_{\alpha}$ returns with $\alpha=0.5$ in initial and adversarial distributions, with standard error bars indicated by black vertical lines. The higher, the better.

Theorems & Definitions (11)

• Example 1: Adversarially Task Robust MAML, AR-MAML
• Example 2: Adversarially Task Robust CNP, AR-CNP
• Definition 1: $(\ell_1, \ell_2)$-bi-Lipschitz Function
• Definition 2: Local Minimax Point
• Remark 1: Entropy of the Generated Task Distribution
• Remark 2: Solution as a Fixed Point
• Theorem 1: Convergence Guarantee
• Theorem 2: Generalization Bound with the Distribution Adversary
• Remark 3: Optimization Order and Solutions
• Definition 3: Global Minimax Point