Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evolutionary Generalized Zero-Shot Learning

Dubing Chen, Chenyi Jiang, Haofeng Zhang

TL;DR

EGZSL introduces a continual online evolution setting for generalized zero-shot learning, enabling a base ZSL model to learn from unlabeled test data streams after deployment. The approach combines pseudo-labeling, an exponential moving average momentum model for global information, and adaptive class/data selection to mitigate forgetting and bias. Experiments on three public benchmarks show consistent improvements over inductive baselines and competitive performance relative to transductive approaches, highlighting practical potential for real-world deployment. A practical evaluation protocol and ablations illustrate the contribution of each component to robust online evolution in ZSL.

Abstract

Attribute-based Zero-Shot Learning (ZSL) has revolutionized the ability of models to recognize new classes not seen during training. However, with the advancement of large-scale models, the expectations have risen. Beyond merely achieving zero-shot generalization, there is a growing demand for universal models that can continually evolve in expert domains using unlabeled data. To address this, we introduce a scaled-down instantiation of this challenge: Evolutionary Generalized Zero-Shot Learning (EGZSL). This setting allows a low-performing zero-shot model to adapt to the test data stream and evolve online. We elaborate on three challenges of this special task, \ie, catastrophic forgetting, initial prediction bias, and evolutionary data class bias. Moreover, we propose targeted solutions for each challenge, resulting in a generic method capable of continuous evolution from a given initial IGZSL model. Experiments on three popular GZSL benchmark datasets demonstrate that our model can learn from the test data stream while other baselines fail. Codes are available at \url{https://github.com/cdb342/EGZSL}.

Evolutionary Generalized Zero-Shot Learning

TL;DR

EGZSL introduces a continual online evolution setting for generalized zero-shot learning, enabling a base ZSL model to learn from unlabeled test data streams after deployment. The approach combines pseudo-labeling, an exponential moving average momentum model for global information, and adaptive class/data selection to mitigate forgetting and bias. Experiments on three public benchmarks show consistent improvements over inductive baselines and competitive performance relative to transductive approaches, highlighting practical potential for real-world deployment. A practical evaluation protocol and ablations illustrate the contribution of each component to robust online evolution in ZSL.

Abstract

Attribute-based Zero-Shot Learning (ZSL) has revolutionized the ability of models to recognize new classes not seen during training. However, with the advancement of large-scale models, the expectations have risen. Beyond merely achieving zero-shot generalization, there is a growing demand for universal models that can continually evolve in expert domains using unlabeled data. To address this, we introduce a scaled-down instantiation of this challenge: Evolutionary Generalized Zero-Shot Learning (EGZSL). This setting allows a low-performing zero-shot model to adapt to the test data stream and evolve online. We elaborate on three challenges of this special task, \ie, catastrophic forgetting, initial prediction bias, and evolutionary data class bias. Moreover, we propose targeted solutions for each challenge, resulting in a generic method capable of continuous evolution from a given initial IGZSL model. Experiments on three popular GZSL benchmark datasets demonstrate that our model can learn from the test data stream while other baselines fail. Codes are available at \url{https://github.com/cdb342/EGZSL}.
Paper Structure (20 sections, 12 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 20 sections, 12 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. EGZSL: Settings and Challenges
  4. Problem Formulation
  5. Challenges Analysis
  6. EGZSL v.s. Similar Settings
  7. Method
  8. Training with Pseudo Labels
  9. Maintenance of Global Information
  10. Class Selection for Stable Training
  11. Data Selection for Effective Training
  12. Overall Objectives
  13. Experiments
  14. Benchmark Protocol
  15. Implementation Details
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Illustration of the proposed EGZSL setting, featuring training with labeled seen class samples at time 0, followed by iterative predictive re-training on randomly divided data from the mixture of seen and unseen test sets in subsequent time steps (ratio of classes in a small batch is undefined). Train: train the current model at each time step with only the data indicated by the arrows. Test: predict the current data with the model obtained in the last time step. Inherit: train the current model based on the model of the last time step.
  • Figure 2: Comparasion of EGZSL with other similar settings in chronological progression. TTA: Test-Time Adaptation; CL: Continual Learning; IGZSL: Inductive Generalized Zero-Shot Learning; TGZSL: Transductive Generalized Zero-Shot Learning; CGZSL: Continual Generalized Zero-Shot Learning. In TTA, seen represents the source domain, and unseen is the target domain. In other settings, the labeled classes that appear in the training set are denoted as seen, and vice versa are unseen. An unknown class means that it can be any class (in seen or unseen classes).
  • Figure 3: (a), (b) A comparison of evolution curves between our approach and ERM (on AWA2, with 100 samples per time step). Our method displays a rise in accuracy over time, while ERM experiences a decline in accuracy.
  • Figure 4: Hyperparameters w.r.t. EGZSL performance on AWA2. (a) Effects of loss balancing coefficient $\lambda$ (Eq. \ref{['eq:all']}). (b) Effects of thresholds $\tau$ in Eq. (\ref{['eq:thre_scale']}). (c), (d) Effects of the smoothing factors $m_1$ and $m_2$ in Eq. (\ref{['eq:ema']}) and (\ref{['eq:thre']}).