Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

KTVIC: A Vietnamese Image Captioning Dataset on the Life Domain

Anh-Cuong Pham, Van-Quang Nguyen, Thi-Hong Vuong, Quang-Thuy Ha

TL;DR

The paper addresses the scarcity of Vietnamese image captioning data beyond narrow domains by introducing KTVIC, a life-domain dataset with 4,327 images and 21,635 Vietnamese captions. It describes an annotation workflow that yields five captions per image and uses RDRSegmenter for Vietnamese tokenization, providing COCO-style splits and rich linguistic statistics. Three baselines—CNN-LSTM, ViT-Transformer, and GRIT—are evaluated, showing transformer-based approaches outperform the CNN baseline and GRIT achieving the strongest results, particularly on CIDEr. The work offers a public resource to advance Vietnamese image captioning research and lays groundwork for future cross-lingual and domain-adaptive captioning in Vietnamese.

Abstract

Image captioning is a crucial task with applications in a wide range of domains, including healthcare and education. Despite extensive research on English image captioning datasets, the availability of such datasets for Vietnamese remains limited, with only two existing datasets. In this study, we introduce KTVIC, a comprehensive Vietnamese Image Captioning dataset focused on the life domain, covering a wide range of daily activities. This dataset comprises 4,327 images and 21,635 Vietnamese captions, serving as a valuable resource for advancing image captioning in the Vietnamese language. We conduct experiments using various deep neural networks as the baselines on our dataset, evaluating them using the standard image captioning metrics, including BLEU, METEOR, CIDEr, and ROUGE. Our findings underscore the effectiveness of the proposed dataset and its potential contributions to the field of image captioning in the Vietnamese context.

KTVIC: A Vietnamese Image Captioning Dataset on the Life Domain

TL;DR

The paper addresses the scarcity of Vietnamese image captioning data beyond narrow domains by introducing KTVIC, a life-domain dataset with 4,327 images and 21,635 Vietnamese captions. It describes an annotation workflow that yields five captions per image and uses RDRSegmenter for Vietnamese tokenization, providing COCO-style splits and rich linguistic statistics. Three baselines—CNN-LSTM, ViT-Transformer, and GRIT—are evaluated, showing transformer-based approaches outperform the CNN baseline and GRIT achieving the strongest results, particularly on CIDEr. The work offers a public resource to advance Vietnamese image captioning research and lays groundwork for future cross-lingual and domain-adaptive captioning in Vietnamese.

Abstract

Image captioning is a crucial task with applications in a wide range of domains, including healthcare and education. Despite extensive research on English image captioning datasets, the availability of such datasets for Vietnamese remains limited, with only two existing datasets. In this study, we introduce KTVIC, a comprehensive Vietnamese Image Captioning dataset focused on the life domain, covering a wide range of daily activities. This dataset comprises 4,327 images and 21,635 Vietnamese captions, serving as a valuable resource for advancing image captioning in the Vietnamese language. We conduct experiments using various deep neural networks as the baselines on our dataset, evaluating them using the standard image captioning metrics, including BLEU, METEOR, CIDEr, and ROUGE. Our findings underscore the effectiveness of the proposed dataset and its potential contributions to the field of image captioning in the Vietnamese context.
Paper Structure (15 sections, 7 figures, 3 tables)

This paper contains 15 sections, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. English Datasets
  4. Vietnamese Datasets
  5. KTVIC: Vietnamese Image Captioning Dataset
  6. Specifications
  7. Annotation Process
  8. Baseline Methods on KTVIC
  9. Baseline 1 with a CNN Encoder and LSTM Decoder
  10. Baseline 2 with a ViT Encoder and Transformer Decoder
  11. Baseline 3 using GRIT Architecture
  12. Experiments on KTVIC
  13. Evaluation Metrics
  14. Experimental Results
  15. Conclusions

Figures (7)

  • Figure 1: An example of image annotation in the KTVIC dataset, where each image is accompanied by five descriptive (segmented) captions.
  • Figure 2: KTVIC word cloud.
  • Figure 3: Frequency of top 50 common words in our KTVIC dataset.
  • Figure 4: Caption length in terms of the number of words in captions
  • Figure 5: The annotation process and the simple interface for human annotators to caption images in the annotation and revising steps.
  • ...and 2 more figures