Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Agent Cooperative Learning for Robust Vision-Language Alignment under OOD Concepts

Philip Xu, Isabel Wagner, Eerke Boiten

TL;DR

A novel Multi-Agent Cooperative Learning framework to address cross-modal alignment collapse in vision-language models when handling out-of-distribution (OOD) concepts, which enables multi-agent feature space name learning and incorporates a context exchange enhanced few-shot learning algorithm.

Abstract

This paper introduces a novel Multi-Agent Cooperative Learning (MACL) framework to address cross-modal alignment collapse in vision-language models when handling out-of-distribution (OOD) concepts. Four core agents, including image, text, name, and coordination agents, collaboratively mitigate modality imbalance through structured message passing. The proposed framework enables multi-agent feature space name learning, incorporates a context exchange enhanced few-shot learning algorithm, and adopts an adaptive dynamic balancing mechanism to regulate inter-agent contributions. Experiments on the VISTA-Beyond dataset demonstrate that MACL significantly improves performance in both few-shot and zero-shot settings, achieving 1-5% precision gains across diverse visual domains.

Multi-Agent Cooperative Learning for Robust Vision-Language Alignment under OOD Concepts

TL;DR

A novel Multi-Agent Cooperative Learning framework to address cross-modal alignment collapse in vision-language models when handling out-of-distribution (OOD) concepts, which enables multi-agent feature space name learning and incorporates a context exchange enhanced few-shot learning algorithm.

Abstract

This paper introduces a novel Multi-Agent Cooperative Learning (MACL) framework to address cross-modal alignment collapse in vision-language models when handling out-of-distribution (OOD) concepts. Four core agents, including image, text, name, and coordination agents, collaboratively mitigate modality imbalance through structured message passing. The proposed framework enables multi-agent feature space name learning, incorporates a context exchange enhanced few-shot learning algorithm, and adopts an adaptive dynamic balancing mechanism to regulate inter-agent contributions. Experiments on the VISTA-Beyond dataset demonstrate that MACL significantly improves performance in both few-shot and zero-shot settings, achieving 1-5% precision gains across diverse visual domains.
Paper Structure (25 sections, 13 equations, 4 figures, 3 tables)

This paper contains 25 sections, 13 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Vision-Language Pre-trained Models
  4. Few-shot and Zero-shot Learning Methods
  5. Out-of-Distribution (OOD) Concept Adaptation Learning
  6. Method
  7. Multi-Agent Collaborative Learning Framework (MACL)
  8. Preliminary
  9. Motivation
  10. Agent Architecture
  11. Message Passing Mechanism
  12. Image Agent
  13. Standard Encoding and Robust Encoding
  14. Text Agent
  15. Standard Encoding
  16. ...and 10 more sections

Figures (4)

  • Figure 1: Cross-Dataset Generalization Results
  • Figure 2: Cross-Modal Alignment Visualization
  • Figure 3: Visual comparison of models' accuracy on various SC domains
  • Figure 4: Visual comparison of models' accuracy on various OOD domains