Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Temporal Image Caption Retrieval Competition -- Description and Results

Jakub Pokrywka, Piotr Wierzchoń, Kornel Weryszko, Krzysztof Jassem

TL;DR

This paper addresses the multimodal challenge of Text-Image retrieval and introduces a novel task that extends the modalities to include temporal data and provides an analysis of the delivered dataset and the process of its creation.

Abstract

Multimodal models, which combine visual and textual information, have recently gained significant recognition. This paper addresses the multimodal challenge of Text-Image retrieval and introduces a novel task that extends the modalities to include temporal data. The Temporal Image Caption Retrieval Competition (TICRC) presented in this paper is based on the Chronicling America and Challenging America projects, which offer access to an extensive collection of digitized historic American newspapers spanning 274 years. In addition to the competition results, we provide an analysis of the delivered dataset and the process of its creation.

Temporal Image Caption Retrieval Competition -- Description and Results

TL;DR

This paper addresses the multimodal challenge of Text-Image retrieval and introduces a novel task that extends the modalities to include temporal data and provides an analysis of the delivered dataset and the process of its creation.

Abstract

Multimodal models, which combine visual and textual information, have recently gained significant recognition. This paper addresses the multimodal challenge of Text-Image retrieval and introduces a novel task that extends the modalities to include temporal data. The Temporal Image Caption Retrieval Competition (TICRC) presented in this paper is based on the Chronicling America and Challenging America projects, which offer access to an extensive collection of digitized historic American newspapers spanning 274 years. In addition to the competition results, we provide an analysis of the delivered dataset and the process of its creation.
Paper Structure (18 sections, 1 equation, 7 figures, 2 tables)

This paper contains 18 sections, 1 equation, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Motivation
  3. Related Work
  4. Temporal language datasets and models
  5. Multimodal vision-language models
  6. Task Definition
  7. Sample Data
  8. Metric
  9. Data Annotation Process
  10. Objects to be annotated
  11. Technical requirements for the image area (bbox)
  12. Rules for text transcription
  13. Data Analysis
  14. Data Split
  15. Datasets Statistics
  16. ...and 3 more sections

Figures (7)

  • Figure 1: Sample picture with a caption above. This picture comes from a newspaper issued dated Jan 11, 1928.
  • Figure 2: Sample input picture
  • Figure 3: Picture selected on the whole page.
  • Figure 4: Testing distribution over the years
  • Figure 6: Word and character per caption statistics in testing dataset
  • ...and 2 more figures