Unlocking Comics: The AI4VA Dataset for Visual Understanding

TL;DR

A novel dataset comprising Franco-Belgian comics from the 1950s annotated for tasks including depth estimation, semantic segmentation, saliency detection, and character identification, which holds promise for computational creativity and offers avenues for the digitization of art and storytelling innovation.

Abstract

In the evolving landscape of deep learning, there is a pressing need for more comprehensive datasets capable of training models across multiple modalities. Concurrently, in digital humanities, there is a growing demand to leverage technology for diverse media adaptation and creation, yet limited by sparse datasets due to copyright and stylistic constraints. Addressing this gap, our paper presents a novel dataset comprising Franco-Belgian comics from the 1950s annotated for tasks including depth estimation, semantic segmentation, saliency detection, and character identification. It consists of two distinct and consistent styles and incorporates object concepts and labels taken from natural images. By including such diverse information across styles, this dataset not only holds promise for computational creativity but also offers avenues for the digitization of art and storytelling innovation. This dataset is a crucial component of the AI4VA Workshop Challenges~\url{https://sites.google.com/view/ai4vaeccv2024}, where we specifically explore depth and saliency. Dataset details at \url{https://github.com/IVRL/AI4VA}.

Figures (8)

• Figure 1: Example images from both comics in our dataset.
• Figure 2: Characters of the respective comic serialization with their ID.
• Figure 3: Annotated panel from 'Yves le loup'.
• Figure 4: Saliency annotations from our dataset
• Figure 5: Number of instances per category in AI4VA, logarithmic scale.