Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Transparency: Exploring LLM Trainings Datasets through Visual Topic Modeling and Semantic Frame

Charles de Dampierre, Andrei Mogoutov, Nicolas Baumard

TL;DR

The paper tackles the opacity of large language model training data by introducing Bunka, a toolkit that combines Topic Modeling Cartography and Semantic Framing to visualize and analyze datasets for transparency and bias. It demonstrates three use cases: (i) visually summarizing prompts with a 2D topic map and cross-embedding comparisons; (ii) filtering Direct Preference Optimization data to accelerate fine-tuning while maintaining performance; and (iii) analyzing biases via semantic frames to quantify content imbalances. Key contributions include the BunkaTopics pipeline (embedding → reduction → clustering → noun-based labeling), a topic-based data curation method for DPO yielding data-efficient gains, and a semantic framing approach that produces scalable bias maps with low compute. Together, these methods offer practical means to improve transparency, data quality, and efficiency in LLM pre-training and fine-tuning, with accessible code for reproducibility.

Abstract

LLMs are now responsible for making many decisions on behalf of humans: from answering questions to classifying things, they have become an important part of everyday life. While computation and model architecture have been rapidly expanding in recent years, the efforts towards curating training datasets are still in their beginnings. This underappreciation of training datasets has led LLMs to create biased and low-quality content. In order to solve that issue, we present Bunka, a software that leverages AI and Cognitive Science to improve the refinement of textual datasets. We show how Topic Modeling coupled with 2-dimensional Cartography can increase the transparency of datasets. We then show how the same Topic Modeling techniques can be applied to Preferences datasets to accelerate the fine-tuning process and increase the capacities of the model on different benchmarks. Lastly, we show how using Frame Analysis can give insights into existing biases in the training corpus. Overall, we argue that we need better tools to explore and increase the quality and transparency of LLMs training datasets.

Towards Transparency: Exploring LLM Trainings Datasets through Visual Topic Modeling and Semantic Frame

TL;DR

The paper tackles the opacity of large language model training data by introducing Bunka, a toolkit that combines Topic Modeling Cartography and Semantic Framing to visualize and analyze datasets for transparency and bias. It demonstrates three use cases: (i) visually summarizing prompts with a 2D topic map and cross-embedding comparisons; (ii) filtering Direct Preference Optimization data to accelerate fine-tuning while maintaining performance; and (iii) analyzing biases via semantic frames to quantify content imbalances. Key contributions include the BunkaTopics pipeline (embedding → reduction → clustering → noun-based labeling), a topic-based data curation method for DPO yielding data-efficient gains, and a semantic framing approach that produces scalable bias maps with low compute. Together, these methods offer practical means to improve transparency, data quality, and efficiency in LLM pre-training and fine-tuning, with accessible code for reproducibility.

Abstract

LLMs are now responsible for making many decisions on behalf of humans: from answering questions to classifying things, they have become an important part of everyday life. While computation and model architecture have been rapidly expanding in recent years, the efforts towards curating training datasets are still in their beginnings. This underappreciation of training datasets has led LLMs to create biased and low-quality content. In order to solve that issue, we present Bunka, a software that leverages AI and Cognitive Science to improve the refinement of textual datasets. We show how Topic Modeling coupled with 2-dimensional Cartography can increase the transparency of datasets. We then show how the same Topic Modeling techniques can be applied to Preferences datasets to accelerate the fine-tuning process and increase the capacities of the model on different benchmarks. Lastly, we show how using Frame Analysis can give insights into existing biases in the training corpus. Overall, we argue that we need better tools to explore and increase the quality and transparency of LLMs training datasets.
Paper Structure (12 sections, 2 equations, 12 figures, 1 table)

This paper contains 12 sections, 2 equations, 12 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related works
  3. Use Case 1: Using Topic Cartography to summarize Prompt
  4. Framework: Topic Modeling Cartography
  5. Method
  6. Use Case 2: Topic Modeling to clean datasets for Direct Preference Optimization (DPO)
  7. Method
  8. Use Case 3: Analyzing bias using Semantic Framing Analysis
  9. Framework: Semantic Frames Cartography
  10. Conclusion and Discussion
  11. Code and data Availability
  12. Annexe

Figures (12)

  • Figure 1: BunkaTopics Architecture
  • Figure 2: Bunka Map of the prompt-collective dataset using the mxbai-embed-large-v1 model
  • Figure 3: Adjusted Rand Index (ARI) between the clustered documents from 4 different embedding models. 1 means high similarity.
  • Figure 4: Process of Direct Preference Optimisation (DPO) with and without Topic Filtering
  • Figure 5: Comparison of results between the DPO-filtered model (Topic Neural Hermes), the model fine-tuned on the full ChatML DPO Pairs (NeuralHermes-2.5-Mistral-7B), and the base model (OpenHermes-2.5-Mistral-7B). Results can be found on the HuggingFace OpenLLM leaderboard.
  • ...and 7 more figures