A Survey on Programmatic Weak Supervision

TL;DR

Programmatic weak supervision (PWS) addresses the labeling bottleneck by encoding diverse weak supervision sources as labeling functions and combining them via label models. It contrasts two-stage and one-stage (joint) workflows, and extends to sequence tagging and general tasks through specialized label models. The survey covers LF types, automatic/interactive/guided LF generation, various label models, end models, and joint models, plus complementary approaches like active learning and semi-supervised learning. It also highlights open challenges in scaling to complex tasks, expanding LF sources, and ensuring ethical, trustworthy AI. Overall, the work synthesizes the mechanisms by which PWS reduces labeling costs while maintaining performance, and points to promising directions for future research and practical adoption.

Abstract

Labeling training data has become one of the major roadblocks to using machine learning. Among various weak supervision paradigms, programmatic weak supervision (PWS) has achieved remarkable success in easing the manual labeling bottleneck by programmatically synthesizing training labels from multiple potentially noisy supervision sources. This paper presents a comprehensive survey of recent advances in PWS. In particular, we give a brief introduction of the PWS learning paradigm, and review representative approaches for each component within PWS's learning workflow. In addition, we discuss complementary learning paradigms for tackling limited labeled data scenarios and how these related approaches can be used in conjunction with PWS. Finally, we identify several critical challenges that remain under-explored in the area to hopefully inspire future research directions in the field.

Figures (1)

• Figure 1: An overview of PWS pipeline zhang2021wrench.