Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ORIS: Online Active Learning Using Reinforcement Learning-based Inclusive Sampling for Robust Streaming Analytics System

Rahul Pandey, Ziwei Zhu, Hemant Purohit

TL;DR

The ORIS method is proposed, a method to perform Online active learning using Reinforcement learning-based Inclusive Sampling of documents for labeling that outperforms traditional baselines in terms of both human labeling performance and the ML model performance.

Abstract

Effective labeled data collection plays a critical role in developing and fine-tuning robust streaming analytics systems. However, continuously labeling documents to filter relevant information poses significant challenges like limited labeling budget or lack of high-quality labels. There is a need for efficient human-in-the-loop machine learning (HITL-ML) design to improve streaming analytics systems. One particular HITL- ML approach is online active learning, which involves iteratively selecting a small set of the most informative documents for labeling to enhance the ML model performance. The performance of such algorithms can get affected due to human errors in labeling. To address these challenges, we propose ORIS, a method to perform Online active learning using Reinforcement learning-based Inclusive Sampling of documents for labeling. ORIS aims to create a novel Deep Q-Network-based strategy to sample incoming documents that minimize human errors in labeling and enhance the ML model performance. We evaluate the ORIS method on emotion recognition tasks, and it outperforms traditional baselines in terms of both human labeling performance and the ML model performance.

ORIS: Online Active Learning Using Reinforcement Learning-based Inclusive Sampling for Robust Streaming Analytics System

TL;DR

The ORIS method is proposed, a method to perform Online active learning using Reinforcement learning-based Inclusive Sampling of documents for labeling that outperforms traditional baselines in terms of both human labeling performance and the ML model performance.

Abstract

Effective labeled data collection plays a critical role in developing and fine-tuning robust streaming analytics systems. However, continuously labeling documents to filter relevant information poses significant challenges like limited labeling budget or lack of high-quality labels. There is a need for efficient human-in-the-loop machine learning (HITL-ML) design to improve streaming analytics systems. One particular HITL- ML approach is online active learning, which involves iteratively selecting a small set of the most informative documents for labeling to enhance the ML model performance. The performance of such algorithms can get affected due to human errors in labeling. To address these challenges, we propose ORIS, a method to perform Online active learning using Reinforcement learning-based Inclusive Sampling of documents for labeling. ORIS aims to create a novel Deep Q-Network-based strategy to sample incoming documents that minimize human errors in labeling and enhance the ML model performance. We evaluate the ORIS method on emotion recognition tasks, and it outperforms traditional baselines in terms of both human labeling performance and the ML model performance.

Paper Structure

This paper contains 29 sections, 5 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Online Active Learning
  4. Deep Reinforcement Learning for Active Learning
  5. Human Factors in Data Labeling
  6. Preliminaries
  7. Online Active Learning
  8. Human Memory and Labeling Error
  9. Approach
  10. Problem Formulation
  11. Overview of ORIS
  12. ORIS Modeling
  13. State
  14. Action
  15. Reward
  16. ...and 14 more sections

Figures (3)

  • Figure 1: Overview of the proposed ORIS architecture. The method aims to sample inclusive documents from streams using a DQN-based agent. Components 1a, 1b, & 1c are described in Sec. \ref{['subsec:oris_m_state']}, while Components 1d, 1e, & 1f are detailed in Sec. \ref{['subsec:oris_m_reward']}.
  • Figure 2: Online Active Learning System Flow.
  • Figure 3: Machine and Human performance comparison over budget exhausted for both Twitter and Reddit Dataset when using the BERT mini and mBERT model for fine-tuning. The shaded color area represents the 95% confidence interval for the five runs. The blue and red shades represent random and uncertainty sampling, respectively. The offline diversity sampling is represented as a dashed yellow line along the x-axis. Both the green color shades represent the proposed ORIS experiments. The $+$ mark represents the ORIS with $\delta=8$, and the $\blacksquare$ mark represents the ORIS with $\delta=16$, respectively.