Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines

Genta Indra Winata, Frederikus Hudi, Patrick Amadeus Irawan, David Anugraha, Rifki Afina Putri, Yutong Wang, Adam Nohejl, Ubaidillah Ariq Prathama, Nedjma Ousidhoum, Afifa Amriani, Anar Rzayev, Anirban Das, Ashmari Pramodya, Aulia Adila, Bryan Wilie, Candy Olivia Mawalim, Ching Lam Cheng, Daud Abolade, Emmanuele Chersoni, Enrico Santus, Fariz Ikhwantri, Garry Kuwanto, Hanyang Zhao, Haryo Akbarianto Wibowo, Holy Lovenia, Jan Christian Blaise Cruz, Jan Wira Gotama Putra, Junho Myung, Lucky Susanto, Maria Angelica Riera Machin, Marina Zhukova, Michael Anugraha, Muhammad Farid Adilazuarda, Natasha Santosa, Peerat Limkonchotiwat, Raj Dabre, Rio Alexander Audino, Samuel Cahyawijaya, Shi-Xiong Zhang, Stephanie Yulia Salim, Yi Zhou, Yinxuan Gui, David Ifeoluwa Adelani, En-Shiun Annie Lee, Shogo Okada, Ayu Purwarianti, Alham Fikri Aji, Taro Watanabe, Derry Tanti Wijaya, Alice Oh, Chong-Wah Ngo

TL;DR

WorldCuisines presents a massive, open-source benchmark for multilingual and multicultural visual question answering focused on global cuisines. It introduces WC-KB (a Wikipedia-derived knowledge base) and WC-VQA (a 1-million-sample VQA corpus across 30 languages) to evaluate dish-name and regional-origin understanding under no-context, contextualized, and adversarial prompts. The work demonstrates that context improves predictions, adversarial prompts degrade performance, and scaling laws persist across model families, with notable gaps in low-resource languages. By providing translations, morphological inflections, and a robust evaluation protocol, this benchmark enables systematic cross-cultural evaluation and future improvements in multicultural VLMs.

Abstract

Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside a training dataset (1 million instances). Our findings show that while VLMs perform better with correct location context, they struggle with adversarial contexts and predicting specific regional cuisines and languages. To support future research, we release a knowledge base with annotated food entries and images along with the VQA data.

WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines

TL;DR

WorldCuisines presents a massive, open-source benchmark for multilingual and multicultural visual question answering focused on global cuisines. It introduces WC-KB (a Wikipedia-derived knowledge base) and WC-VQA (a 1-million-sample VQA corpus across 30 languages) to evaluate dish-name and regional-origin understanding under no-context, contextualized, and adversarial prompts. The work demonstrates that context improves predictions, adversarial prompts degrade performance, and scaling laws persist across model families, with notable gaps in low-resource languages. By providing translations, morphological inflections, and a robust evaluation protocol, this benchmark enables systematic cross-cultural evaluation and future improvements in multicultural VLMs.

Abstract

Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside a training dataset (1 million instances). Our findings show that while VLMs perform better with correct location context, they struggle with adversarial contexts and predicting specific regional cuisines and languages. To support future research, we release a knowledge base with annotated food entries and images along with the VQA data.

Paper Structure

This paper contains 32 sections, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. $\textcolor{black}{WorldCuisines}$
  3. Overview
  4. $\textcolor{black}{WC-KB}$.
  5. $\textcolor{black}{WC-VQA}$.
  6. $\textcolor{black}{WC-KB}$ Construction
  7. Dish Selection
  8. Metadata Annotation
  9. Quality Assurance
  10. Data Compilation
  11. VQA Generation
  12. Dish Names Similarity Search
  13. Question and Context Construction
  14. Multiple Language Translation
  15. Question and Context.
  16. ...and 17 more sections

Figures (6)

  • Figure 1: Images of stuffed pasta and dumplings from our dataset showcase a similar culinary concept across different cultures: wrapping meat, dairy (such as cheese), or vegetables in dough. These dishes can be prepared in various ways, including pan-frying, deep-frying, steaming, or boiling.
  • Figure 2: $\textcolor{black}{WC-VQA}$ in $\textcolor{black}{WorldCuisines}$ comprises two primary tasks: (1) predicting dish names and (2) predicting regional cuisines. Task 1 is further divided into three subtasks: (a) no-context, (b) contextualized, and (c) adversarial. We also include two answer types: multiple-choice question (MCQ) and open-ended question (OEQ).
  • Figure 3: $\textcolor{black}{WorldCuisines}$ distribution of food entries by country in the World Map. The food entries are distributed across 189 countries, with the highest concentration found in Asia, Europe, and North America. There are also some entries from the continents of Africa, Oceania, and Central and South America.
  • Figure 4: Countries by number of assigned dishes, showing the top 50 countries.
  • Figure 5: Accuracy (%) categorized by language (left), language vitality (center), and language family (right). We classify the language vitality by following the classification proposed by joshi2020state.
  • ...and 1 more figures