Automatic Histograms: Leveraging Language Models for Text Dataset Exploration

TL;DR

This work presents AutoHistograms, a visualization tool that uses LLMs and embeddings to automatically identify dataset-specific attributes in unstructured text and visualize their distributions as histograms, with real-time histogram creation for in-the-loop exploration. The system combines a preprocessing pipeline (entity extraction, hierarchical clustering, and LLM-based labeling) with an interactive UI supporting exact and semantic search. An observational study with 10 data workers demonstrates rapid onboarding, effective hypothesis testing, and use cases in safety, fairness, and data quality. The paper contributes to the growing field of LLM-assisted sensemaking by enabling dataset-specific distribution discovery and interactive, real-time data exploration.

Abstract

Making sense of unstructured text datasets is perennially difficult, yet increasingly relevant with Large Language Models. Data workers often rely on dataset summaries, especially distributions of various derived features. Some features, like toxicity or topics, are relevant to many datasets, but many interesting features are domain specific: instruments and genres for a music dataset, or diseases and symptoms for a medical dataset. Accordingly, data workers often run custom analyses for each dataset, which is cumbersome and difficult. We present AutoHistograms, a visualization tool leveragingLLMs. AutoHistograms automatically identifies relevant features, visualizes them with histograms, and allows the user to interactively query the dataset for categories of entities and create new histograms. In a user study with 10 data workers (n=10), we observe that participants can quickly identify insights and explore the data using AutoHistograms, and conceptualize a broad range of applicable use cases. Together, this tool and user study contributeto the growing field of LLM-assisted sensemaking tools.

Figures (4)

• Figure 1: The processing steps for automatically creating histograms from a dataset
• Figure 2: A search interface that supports exact or semantic search.
• Figure 3: The steps for creating a new histogram on-the-fly. The user types in a category of entities they wish to find in the dataset (e.g., "sexually transmitted diseases"). Using LLMs and embeddings, we surface entities in this category, which the user can confirm to create a new histogram.
• Figure 4: When the histogram bucket for an entity is selected, the data table is filtered to only examples that contain that entity.