Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLMs as Cultural Archives: Cultural Commonsense Knowledge Graph Extraction

Junior Cedric Tonga, Chen Cecilia Liu, Iryna Gurevych, Fajri Koto

TL;DR

The paper investigates using large language models as cultural archives to extract a Cultural Commonsense Knowledge Graph (CCKG) that captures multilingual, culture-specific inferential chains. It introduces an iterative, prompt-based framework to construct CCKG across languages, with a formal graph representation and two-stage construction (Initial Generation and Iterative Expansion). Through cross-country experiments (England, China, Egypt, Indonesia, Japan) and downstream tasks (cultural commonsense reasoning and story generation), it demonstrates that English extractions tend to be more coherent, while native languages yield richer cultural detail; augmenting smaller LLMs with CCKG improves cultural reasoning and narrative generation, though there are limits and potential biases. The work highlights both the promise and constraints of treating LLMs as cultural archives and shows that chain-structured cultural knowledge can serve as a practical substrate for culturally grounded NLP, while urging careful handling of prompts and cultural bias. Overall, CCKG provides a scalable, multilingual approach to surface and utilize culturally grounded, inferential knowledge for NLP applications.

Abstract

Large language models (LLMs) encode rich cultural knowledge learned from diverse web-scale data, offering an unprecedented opportunity to model cultural commonsense at scale. Yet this knowledge remains mostly implicit and unstructured, limiting its interpretability and use. We present an iterative, prompt-based framework for constructing a Cultural Commonsense Knowledge Graph (CCKG) that treats LLMs as cultural archives, systematically eliciting culture-specific entities, relations, and practices and composing them into multi-step inferential chains across languages. We evaluate CCKG on five countries with human judgments of cultural relevance, correctness, and path coherence. We find that the cultural knowledge graphs are better realized in English, even when the target culture is non-English (e.g., Chinese, Indonesian, Arabic), indicating uneven cultural encoding in current LLMs. Augmenting smaller LLMs with CCKG improves performance on cultural reasoning and story generation, with the largest gains from English chains. Our results show both the promise and limits of LLMs as cultural technologies and that chain-structured cultural knowledge is a practical substrate for culturally grounded NLP.

LLMs as Cultural Archives: Cultural Commonsense Knowledge Graph Extraction

TL;DR

The paper investigates using large language models as cultural archives to extract a Cultural Commonsense Knowledge Graph (CCKG) that captures multilingual, culture-specific inferential chains. It introduces an iterative, prompt-based framework to construct CCKG across languages, with a formal graph representation and two-stage construction (Initial Generation and Iterative Expansion). Through cross-country experiments (England, China, Egypt, Indonesia, Japan) and downstream tasks (cultural commonsense reasoning and story generation), it demonstrates that English extractions tend to be more coherent, while native languages yield richer cultural detail; augmenting smaller LLMs with CCKG improves cultural reasoning and narrative generation, though there are limits and potential biases. The work highlights both the promise and constraints of treating LLMs as cultural archives and shows that chain-structured cultural knowledge can serve as a practical substrate for culturally grounded NLP, while urging careful handling of prompts and cultural bias. Overall, CCKG provides a scalable, multilingual approach to surface and utilize culturally grounded, inferential knowledge for NLP applications.

Abstract

Large language models (LLMs) encode rich cultural knowledge learned from diverse web-scale data, offering an unprecedented opportunity to model cultural commonsense at scale. Yet this knowledge remains mostly implicit and unstructured, limiting its interpretability and use. We present an iterative, prompt-based framework for constructing a Cultural Commonsense Knowledge Graph (CCKG) that treats LLMs as cultural archives, systematically eliciting culture-specific entities, relations, and practices and composing them into multi-step inferential chains across languages. We evaluate CCKG on five countries with human judgments of cultural relevance, correctness, and path coherence. We find that the cultural knowledge graphs are better realized in English, even when the target culture is non-English (e.g., Chinese, Indonesian, Arabic), indicating uneven cultural encoding in current LLMs. Augmenting smaller LLMs with CCKG improves performance on cultural reasoning and story generation, with the largest gains from English chains. Our results show both the promise and limits of LLMs as cultural technologies and that chain-structured cultural knowledge is a practical substrate for culturally grounded NLP.
Paper Structure (46 sections, 3 equations, 10 figures, 15 tables, 1 algorithm)

This paper contains 46 sections, 3 equations, 10 figures, 15 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Knowledge Bases for Commonsense Reasoning.
  4. Evaluating Cultural Commonsense Knowledge.
  5. Cultural Commonsense Knowledge Graph (CCKG)
  6. Preliminaries
  7. Knowledge Graph Construction
  8. Initial Generation.
  9. Iterative Expansion.
  10. Experiments
  11. CCKG Extraction
  12. Topic Taxonomy and Country Selection.
  13. Evaluation Setup.
  14. Preliminary Experiments.
  15. Extraction and Evaluation.
  16. ...and 31 more sections

Figures (10)

  • Figure 1: Application of our framework for constructing a partial Cultural Commonsense Knowledge Graph (CCKG) capturing culturally grounded reasoning about breakfast in Indonesia. Given an input prompt specifying the subtopic, language, country, and task-specific constraints, GPT-4o generates English if--then commonsense assertions $(\textit{action}_i, \textit{relation}, \textit{action}_j)$ to form an initial knowledge base (KB). Assertions with relations (xNext, oNext) are iteratively expanded by re-prompting GPT-4o to generate intermediate action expansions that decompose $\textit{action}_i$ into finer-grained steps leading to $\textit{action}_j$ and forward actions occurring after $\textit{action}_j$. In this example, only the first assertion in the expansion list is expanded for a single iteration. The resulting assertions are added to the KB, post-processed and composed into the final CCKG subgraph.
  • Figure 2: Relative improvement from +CCKG over the baseline in Native vs. English story generation. Bars show percentage lift in average human scores (Cultural relevance, Fluency, Coherence); numbers above bars indicate gains in percentage points.
  • Figure 3: Performance of Llama3.3-70B-IT and GPT-4o on initial generation data to determine the optimal model for CCKG generation.
  • Figure 4: Average scores per evaluation metric for each country and model.
  • Figure 5: Prompt used in the initial generation step of CCKG. The variables sub_topic, location, and language are replaced with their corresponding values (sub-topic, country, and language), expressed in English when the KB is generated in English and in the native language otherwise.
  • ...and 5 more figures