A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition
Zihan Wang, Ziqi Zhao, Yougang Lyu, Zhumin Chen, Maarten de Rijke, Zhaochun Ren
TL;DR
This paper tackles zero-shot NER under no labeled data by addressing two core challenges: contextual correlations around entities and the unreliable use of demonstrations. It introduces CMAS, a cooperative four-agent framework consisting of a self-annotator, a type-related feature (TRF) extractor, a demonstration discriminator with a self-reflection mechanism, and an overall predictor, enabling two-subtask modeling of entities and TRFs and a selective, context-aware use of demonstrations. Through extensive zero-shot and few-shot experiments on six benchmarks (general and domain-specific), CMAS achieves state-of-the-art results and demonstrates robustness across LLM backbones and varying demonstration counts. Ablation and error analyses show the crucial roles of TRF extraction and demonstration discrimination, with potential gains from external syntactic tools, highlighting the practical impact of cooperative multi-agent prompting for information extraction tasks with limited supervision.
Abstract
Zero-shot named entity recognition (NER) aims to develop entity recognition systems from unannotated text corpora. This task presents substantial challenges due to minimal human intervention. Recent work has adapted large language models (LLMs) for zero-shot NER by crafting specialized prompt templates. It advances model self-learning abilities by incorporating self-annotated demonstrations. However, two important challenges persist: (i) Correlations between contexts surrounding entities are overlooked, leading to wrong type predictions or entity omissions. (ii) The indiscriminate use of task demonstrations, retrieved through shallow similarity-based strategies, severely misleads LLMs during inference. In this paper, we introduce the cooperative multi-agent system (CMAS), a novel framework for zero-shot NER that uses the collective intelligence of multiple agents to address the challenges outlined above. CMAS has four main agents: (i) a self-annotator, (ii) a type-related feature (TRF) extractor, (iii) a demonstration discriminator, and (iv) an overall predictor. To explicitly capture correlations between contexts surrounding entities, CMAS reformulates NER into two subtasks: recognizing named entities and identifying entity type-related features within the target sentence. To enable controllable utilization of demonstrations, a demonstration discriminator is established to incorporate the self-reflection mechanism, automatically evaluating helpfulness scores for the target sentence. Experimental results show that CMAS significantly improves zero-shot NER performance across six benchmarks, including both domain-specific and general-domain scenarios. Furthermore, CMAS demonstrates its effectiveness in few-shot settings and with various LLM backbones.