Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RAG-based Crowdsourcing Task Decomposition via Masked Contrastive Learning with Prompts

Jing Yang, Xiao Wang, Yu Zhao, Yuhang Liu, Fei-Yue Wang

TL;DR

The paper tackles the challenge of robust task decomposition in crowdsourcing where pretrained language models can hallucinate or rely on outdated knowledge. It introduces PBCT, a prompt-based, retrieval-augmented framework that redefines task decomposition as event detection and combines a prompt-based trigger detector, a prototypical network classifier, a trigger-attentive sentinel, and masked contrastive learning. Experiments on ACE 2005 and FewEvent (with a PCB case study) show strong performance in both supervised and zero-shot settings, with ablations confirming the importance of semantic initialization, contextual-trigger balancing, and contrastive learning. The work demonstrates practical impact by enabling more reliable TD across professional domains, highlighting PBCT’s potential for robust open-world crowdsourcing applications.

Abstract

Crowdsourcing is a critical technology in social manufacturing, which leverages an extensive and boundless reservoir of human resources to handle a wide array of complex tasks. The successful execution of these complex tasks relies on task decomposition (TD) and allocation, with the former being a prerequisite for the latter. Recently, pre-trained language models (PLMs)-based methods have garnered significant attention. However, they are constrained to handling straightforward common-sense tasks due to their inherent restrictions involving limited and difficult-to-update knowledge as well as the presence of hallucinations. To address these issues, we propose a retrieval-augmented generation-based crowdsourcing framework that reimagines TD as event detection from the perspective of natural language understanding. However, the existing detection methods fail to distinguish differences between event types and always depend on heuristic rules and external semantic analyzing tools. Therefore, we present a Prompt-Based Contrastive learning framework for TD (PBCT), which incorporates a prompt-based trigger detector to overcome dependence. Additionally, trigger-attentive sentinel and masked contrastive learning are introduced to provide varying attention to trigger and contextual features according to different event types. Experiment results demonstrate the competitiveness of our method in both supervised and zero-shot detection. A case study on printed circuit board manufacturing is showcased to validate its adaptability to unknown professional domains.

RAG-based Crowdsourcing Task Decomposition via Masked Contrastive Learning with Prompts

TL;DR

The paper tackles the challenge of robust task decomposition in crowdsourcing where pretrained language models can hallucinate or rely on outdated knowledge. It introduces PBCT, a prompt-based, retrieval-augmented framework that redefines task decomposition as event detection and combines a prompt-based trigger detector, a prototypical network classifier, a trigger-attentive sentinel, and masked contrastive learning. Experiments on ACE 2005 and FewEvent (with a PCB case study) show strong performance in both supervised and zero-shot settings, with ablations confirming the importance of semantic initialization, contextual-trigger balancing, and contrastive learning. The work demonstrates practical impact by enabling more reliable TD across professional domains, highlighting PBCT’s potential for robust open-world crowdsourcing applications.

Abstract

Crowdsourcing is a critical technology in social manufacturing, which leverages an extensive and boundless reservoir of human resources to handle a wide array of complex tasks. The successful execution of these complex tasks relies on task decomposition (TD) and allocation, with the former being a prerequisite for the latter. Recently, pre-trained language models (PLMs)-based methods have garnered significant attention. However, they are constrained to handling straightforward common-sense tasks due to their inherent restrictions involving limited and difficult-to-update knowledge as well as the presence of hallucinations. To address these issues, we propose a retrieval-augmented generation-based crowdsourcing framework that reimagines TD as event detection from the perspective of natural language understanding. However, the existing detection methods fail to distinguish differences between event types and always depend on heuristic rules and external semantic analyzing tools. Therefore, we present a Prompt-Based Contrastive learning framework for TD (PBCT), which incorporates a prompt-based trigger detector to overcome dependence. Additionally, trigger-attentive sentinel and masked contrastive learning are introduced to provide varying attention to trigger and contextual features according to different event types. Experiment results demonstrate the competitiveness of our method in both supervised and zero-shot detection. A case study on printed circuit board manufacturing is showcased to validate its adaptability to unknown professional domains.
Paper Structure (20 sections, 15 equations, 9 figures, 4 tables)

This paper contains 20 sections, 15 equations, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. RAG-based Crowdsourcing
  3. Task Decomposition
  4. Problem Statement
  5. Model Architecture
  6. Prompt-based Trigger Detector
  7. Prototypical Network Classifier
  8. Trigger-Attentive Sentinel
  9. Masked Contrastive Learning
  10. Supervised Loss
  11. Experiments
  12. Datasets and Baselines
  13. Training Settings and Evaluation Metrics
  14. Results and analysis
  15. Ablation study
  16. ...and 5 more sections

Figures (9)

  • Figure 1: The workflow of crowdsouring.
  • Figure 2: The illustration of Task decomposition. The numbers represents the order in which subtasks are carried out.
  • Figure 3: RAG-based crowdsourcing
  • Figure 4: Two typical instances taken from the ACE 2005 benchmark.
  • Figure 5: The overall architecture of PBCT, which consists of four modules: A prompt-based trigger detector, a prototypical network classifier, a trigger-attentive sentinel and masked contrastive learning.
  • ...and 4 more figures