TempEval-3: Evaluating Events, Time Expressions, and Temporal Relations
Naushad UzZaman, Hector Llorens, James Allen, Leon Derczynski, Marc Verhagen, James Pustejovsky
TL;DR
TempEval-3 presents a comprehensive temporal information processing shared task for SemEval-2013, expanding from previous editions with three interdependent tasks (TIMEX3 expression extraction/normalization, event extraction, and temporal relation annotation) and a much larger mixed silver/gold dataset. It introduces a raw-text relation task, a full TimeML relation set, and a single temporal-awareness evaluation metric to rank systems. A hybrid data-creation strategy combines automated annotation from multiple systems with selective human review, enabling both large-scale training and rigorous evaluation. This work aims to push forward temporal annotation research by providing standardized data, unified evaluation, and practical release timelines for the NLP community.
Abstract
We describe the TempEval-3 task which is currently in preparation for the SemEval-2013 evaluation exercise. The aim of TempEval is to advance research on temporal information processing. TempEval-3 follows on from previous TempEval events, incorporating: a three-part task structure covering event, temporal expression and temporal relation extraction; a larger dataset; and single overall task quality scores.