Reinforced Interactive Continual Learning via Real-time Noisy Human Feedback

TL;DR

RiCL tackles the challenge of interactive continual learning in dynamic, real-world environments by learning from streaming real-time human feedback while robustly handling label noise. It integrates three components: a temporal consistency-aware purifier to filter streaming data, an interaction-aware direct preference optimization to align model behavior with human intent, and a noise-resistant contrastive learning module to learn robust representations from noisy data. The approach is validated on FewRel and TACRED under realistic noise and task-overlap settings, where RiCL outperforms strong online continual learning and noisy-label baselines, while exhibiting resilience to task order and increasing noise. This work advances practical lifelong learning for open-world AI systems by bridging human-in-the-loop feedback with robust, real-time adaptation.

Abstract

This paper introduces an interactive continual learning paradigm where AI models dynamically learn new skills from real-time human feedback while retaining prior knowledge. This paradigm distinctively addresses two major limitations of traditional continual learning: (1) dynamic model updates using streaming, real-time human-annotated data, rather than static datasets with fixed labels, and (2) the assumption of clean labels, by explicitly handling the noisy feedback common in real-world interactions. To tackle these problems, we propose RiCL, a Reinforced interactive Continual Learning framework leveraging Large Language Models (LLMs) to learn new skills effectively from dynamic feedback. RiCL incorporates three key components: a temporal consistency-aware purifier to automatically discern clean from noisy samples in data streams; an interaction-aware direct preference optimization strategy to align model behavior with human intent by reconciling AI-generated and human-provided feedback; and a noise-resistant contrastive learning module that captures robust representations by exploiting inherent data relationships, thus avoiding reliance on potentially unreliable labels. Extensive experiments on two benchmark datasets (FewRel and TACRED), contaminated with realistic noise patterns, demonstrate that our RiCL approach substantially outperforms existing combinations of state-of-the-art online continual learning and noisy-label learning methods.

Figures (4)

• Figure 1: The process of Interactive Continual Learning.
• Figure 2: The framework of our Reinforced interactive Continual Learning (RiCL).
• Figure 3: AP and AF on Tacred and Fewrel across distinct task orders.
• Figure 4: AP and AF on Tacred and Fewrel under 20% and 40% symmetric label noise.