Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Injecting Knowledge from Social Science Journals to Improve Indonesian Cultural Understanding by LLMs

Adimulya Kartiyasa, Bao Gia Cao, Boyang Li

TL;DR

This work tackles the underrepresentation of Indonesian culture in LLM knowledge by constructing IndoSoSci, a large-scale corpus of Indonesian social science journal passages, and introducing a retrieval-augmented generation pipeline that extracts culture-related facts and uses model-generated hypothetical documents as retrieval queries. The approach yields strong gains on the IndoCulture benchmark, with a best 81.4% accuracy when combining IndoSoSci facts with Indonesian Wikipedia, setting a new state of the art. Key contributions include the IndoSoSci dataset, the fact-extraction methodology, and the demonstration that RAG with a domain-specific scholarly corpus can meaningfully enhance cultural understanding in multilingual Southeast Asian LLMs. The findings indicate that integrating diverse sources—especially native social science literature—can complement Wikipedia and improve cultural literacy in LLMs, with practical implications for more culturally aware AI tools in Indonesia and similar regions.

Abstract

Recently there have been intensifying efforts to improve the understanding of Indonesian cultures by large language models (LLMs). An attractive source of cultural knowledge that has been largely overlooked is local journals of social science, which likely contain substantial cultural studies from a native perspective. We present a novel text dataset of journal article passages, created from 151 open-source Indonesian social science journals, called IndoSoSci. We demonstrate an effective recipe for injecting Indonesian cultural knowledge therein into LLMs: extracting the facts related to Indonesian culture, and apply retrieval-augmented generation (RAG) with LLM-generated hypothetical documents as queries during retrieval. The proposed recipe yields strong performance gains over several strong baselines on the IndoCulture benchmark. Additionally, by combining IndoSoSci with Indonesian Wikipedia, we set a new state-of-the-art accuracy on the IndoCulture benchmark.

Injecting Knowledge from Social Science Journals to Improve Indonesian Cultural Understanding by LLMs

TL;DR

This work tackles the underrepresentation of Indonesian culture in LLM knowledge by constructing IndoSoSci, a large-scale corpus of Indonesian social science journal passages, and introducing a retrieval-augmented generation pipeline that extracts culture-related facts and uses model-generated hypothetical documents as retrieval queries. The approach yields strong gains on the IndoCulture benchmark, with a best 81.4% accuracy when combining IndoSoSci facts with Indonesian Wikipedia, setting a new state of the art. Key contributions include the IndoSoSci dataset, the fact-extraction methodology, and the demonstration that RAG with a domain-specific scholarly corpus can meaningfully enhance cultural understanding in multilingual Southeast Asian LLMs. The findings indicate that integrating diverse sources—especially native social science literature—can complement Wikipedia and improve cultural literacy in LLMs, with practical implications for more culturally aware AI tools in Indonesia and similar regions.

Abstract

Recently there have been intensifying efforts to improve the understanding of Indonesian cultures by large language models (LLMs). An attractive source of cultural knowledge that has been largely overlooked is local journals of social science, which likely contain substantial cultural studies from a native perspective. We present a novel text dataset of journal article passages, created from 151 open-source Indonesian social science journals, called IndoSoSci. We demonstrate an effective recipe for injecting Indonesian cultural knowledge therein into LLMs: extracting the facts related to Indonesian culture, and apply retrieval-augmented generation (RAG) with LLM-generated hypothetical documents as queries during retrieval. The proposed recipe yields strong performance gains over several strong baselines on the IndoCulture benchmark. Additionally, by combining IndoSoSci with Indonesian Wikipedia, we set a new state-of-the-art accuracy on the IndoCulture benchmark.
Paper Structure (21 sections, 3 figures, 9 tables)

This paper contains 21 sections, 3 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Creating the IndoSoSci Dataset
  5. Cultural Facts Extraction from Academic Text
  6. RAG with Hypothetical Documents
  7. Experiments
  8. Setup
  9. RAG Extracted Facts from Social Science Journals
  10. RAG with Mixture of Extracted Journal Facts and Wikipedia
  11. Conclusion
  12. Risks
  13. Prompt for Fact Extraction
  14. Example Fact Extraction Result from Journal
  15. Example Fact Extraction Result from Wikipedia
  16. ...and 6 more sections

Figures (3)

  • Figure 1: The proportions of social science topics covered by the IndoSoSci dataset.
  • Figure 2: A word cloud of the frequent phrases in the academic text extracted from the regions labeled with main title, section title, abstract, text, and list.
  • Figure 3: A word cloud of the frequent phrases in the text of the cultural facts extracted.