How to Enable Effective Cooperation Between Humans and NLP Models: A Survey of Principles, Formalizations, and Beyond

TL;DR

The paper tackles the problem of formalizing effective human-model cooperation in NLP by defining clear terms and principles, and by proposing a unified taxonomy of three cooperative forms: sequential, triage-based, and joint, each with distinct role frameworks. It contributes a principled overview, a taxonomy with practical illustrations, and a discussion of open technical and social challenges, aiming to standardize evaluation and guide future research. The authors highlight gaps in benchmarks, uncertainty estimation, model coordination, and ethical/regulatory considerations, arguing for robust, trustworthy integration of humans and LLM-based agents. The work lays a foundational blueprint for designing and evaluating cooperative AI systems that leverage complementary human and model strengths across data annotation, information seeking, and beyond.

Abstract

With the advancement of large language models (LLMs), intelligent models have evolved from mere tools to autonomous agents with their own goals and strategies for cooperating with humans. This evolution has birthed a novel paradigm in NLP, i.e., human-model cooperation, that has yielded remarkable progress in numerous NLP tasks in recent years. In this paper, we take the first step to present a thorough review of human-model cooperation, exploring its principles, formalizations, and open challenges. In particular, we introduce a new taxonomy that provides a unified perspective to summarize existing approaches. Also, we discuss potential frontier areas and their corresponding challenges. We regard our work as an entry point, paving the way for more breakthrough research in this regard.

Figures (8)

• Figure 1: Unified taxonomy for categorizing the formalization of human-model cooperation. We also introduce two role frameworks, defining how the cooperators contribute to the overall task.
• Figure 2: Two types of sequential cooperation based on whether the model or the human, respectively, takes on the role of the assistor.
• Figure 3: Triage-based cooperation allocates tasks based on capabilities of two parties, achieved by two types of allocators. It follows equal-partnership framework where cooperators are responsible for their own tasks.
• Figure 4: Joint cooperation combines outputs of both parties. Both parties take responsibility for the final outputs.
• Figure 5: Overview of the survey.