Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Primitive Relations for Compositional Zero-Shot Learning

Insu Lee, Jiseob Kim, Kyuhong Shim, Byonghyo Shim

TL;DR

This work tackles Compositional Zero-Shot Learning (CZSL), where models must recognize unseen state–object pairs. It introduces Learning Primitive Relations (LPR), a cross-attention–based framework with three branches (com, sor, osr) that probabilistically model state–object relationships to infer unseen compositions. By leveraging CLIP features and adapters, LPR achieves state-of-the-art results on MIT-States, UT-Zappos, and C-GQA in both closed-world and open-world CZSL, with ablations confirming the benefit of the triple-branch design. Overall, the approach advances compositional generalization by moving beyond independent prediction of states and objects to relational reasoning between primitives, with practical impact for robust zero-shot generalization in vision-language tasks.

Abstract

Compositional Zero-Shot Learning (CZSL) aims to identify unseen state-object compositions by leveraging knowledge learned from seen compositions. Existing approaches often independently predict states and objects, overlooking their relationships. In this paper, we propose a novel framework, learning primitive relations (LPR), designed to probabilistically capture the relationships between states and objects. By employing the cross-attention mechanism, LPR considers the dependencies between states and objects, enabling the model to infer the likelihood of unseen compositions. Experimental results demonstrate that LPR outperforms state-of-the-art methods on all three CZSL benchmark datasets in both closed-world and open-world settings. Through qualitative analysis, we show that LPR leverages state-object relationships for unseen composition prediction.

Learning Primitive Relations for Compositional Zero-Shot Learning

TL;DR

This work tackles Compositional Zero-Shot Learning (CZSL), where models must recognize unseen state–object pairs. It introduces Learning Primitive Relations (LPR), a cross-attention–based framework with three branches (com, sor, osr) that probabilistically model state–object relationships to infer unseen compositions. By leveraging CLIP features and adapters, LPR achieves state-of-the-art results on MIT-States, UT-Zappos, and C-GQA in both closed-world and open-world CZSL, with ablations confirming the benefit of the triple-branch design. Overall, the approach advances compositional generalization by moving beyond independent prediction of states and objects to relational reasoning between primitives, with practical impact for robust zero-shot generalization in vision-language tasks.

Abstract

Compositional Zero-Shot Learning (CZSL) aims to identify unseen state-object compositions by leveraging knowledge learned from seen compositions. Existing approaches often independently predict states and objects, overlooking their relationships. In this paper, we propose a novel framework, learning primitive relations (LPR), designed to probabilistically capture the relationships between states and objects. By employing the cross-attention mechanism, LPR considers the dependencies between states and objects, enabling the model to infer the likelihood of unseen compositions. Experimental results demonstrate that LPR outperforms state-of-the-art methods on all three CZSL benchmark datasets in both closed-world and open-world settings. Through qualitative analysis, we show that LPR leverages state-object relationships for unseen composition prediction.
Paper Structure (14 sections, 2 equations, 5 figures, 3 tables)

This paper contains 14 sections, 2 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Learning Primitive Relations
  3. CZSL Formulation
  4. CLIP Feature Extraction for Compositional Classification
  5. Role of LPR Branches
  6. LPR Training and Inference
  7. Experimental Results
  8. Setup
  9. CZSL Performance
  10. Ablation Study
  11. Sensitivity to Hyperparameters
  12. Qualitative Analysis
  13. Related Work
  14. Conclusion

Figures (5)

  • Figure 1: Illustration of the conventional and proposed CZSL approaches. While the conventional methods assume independence between states and objects, the proposed LPR learns probabilistic relationships between primitives.
  • Figure 2: Overview of the proposed LPR. LPR consists of three branches, namely compositional (cyan, com), state object relation (purple, sor), and object state relation (yellow, osr) branch. Best viewed in color.
  • Figure 3: Architecture of the state object relation (sor) branch.
  • Figure 4: Effect of $\alpha$ during inference on MIT-States in the open-world setting.
  • Figure 5: Composition classification results from the three different branches. 'Prediction' indicates the final results. Green and red words indicate correct and incorrect predictions, respectively.