Continual Learning for Temporal-Sensitive Question Answering

TL;DR

This work introduces Continual Learning for Temporal Sensitive Question Answering (CLTSQA), a setting where temporal information evolves and models must continually adapt without forgetting. It contributes CLTSQA-Data, a time-ordered dataset of 50,000 questions across 5 subsets derived from TimeQA, and a model-agnostic CLTSQA-Framework combining Temporal Memory Replay (TMR) and Temporal Contrastive Learning (TCL). Empirical results with FiD and BigBird show that the framework improves performance on both current and earlier time periods, with TMR driving stronger retention and TCL providing additional gains through temporal discrimination. The findings offer a practical pathway toward robust, up-to-date temporal QA in dynamic information environments, and point to further exploration of datasets and methods for CLTSQA.

Abstract

In this study, we explore an emerging research area of Continual Learning for Temporal Sensitive Question Answering (CLTSQA). Previous research has primarily focused on Temporal Sensitive Question Answering (TSQA), often overlooking the unpredictable nature of future events. In real-world applications, it's crucial for models to continually acquire knowledge over time, rather than relying on a static, complete dataset. Our paper investigates strategies that enable models to adapt to the ever-evolving information landscape, thereby addressing the challenges inherent in CLTSQA. To support our research, we first create a novel dataset, divided into five subsets, designed specifically for various stages of continual learning. We then propose a training framework for CLTSQA that integrates temporal memory replay and temporal contrastive learning. Our experimental results highlight two significant insights: First, the CLTSQA task introduces unique challenges for existing models. Second, our proposed framework effectively navigates these challenges, resulting in improved performance.

Figures (7)

• Figure 1: The difference of training process between TSQA and CLTSQA. While TSQA assumes the availability of the whole training dataset, CLTSQA requires the model to keep ingesting up-to-date new knowledge.
• Figure 2: Examples of CLTSQA-Data. The above part shows the dataset divided based on time intervals. In the bottom part, the left side represents the context and the right side represents the corresponding question-target pairs.
• Figure 3: An overview of the CLTSQA task with our framework. The above figure illustrates the sequential learning of different subsets. The below figure represents our approach of loading the pre-trained weights of the previous model for the next model, while incorporating temporal memory replay and temporal contrastive learning.
• Figure 4: Illustration of temporal contrastive learning, including generation process of contrastive and similar questions as well as model's learning process.
• Figure 5: The testing performance of "FiD-Baseline" and "FiD-CLTSQA" models in different training stages. The proposed framework effectively helps the FiD model to retain the performance on old subsets throughout the training process.