What LLMs Think When You Don't Tell Them What to Think About?

TL;DR

The paper investigates what LLMs generate when confronted with minimally constrained, topic-neutral prompts, aiming to reveal near-unconstrained top-of-mind content and improve model monitoring. It introduces a three-stage methodology—text generation with topic-neutral seeds, elimination of degenerate text, and semantic labeling with embeddings—and applies it to sixteen open-source LLMs across four families, producing 256k samples. Findings show broad semantic coverage but clear family-specific topical preferences and varying depth levels, with degenerate text patterns offering additional behavioral signals and privacy concerns. The work provides a valuable dataset and reproducible code, enabling further auditing and fingerprinting of LLMs in near-unconstrained settings.

Abstract

Characterizing the behavior of large language models (LLMs) across diverse settings is critical for reliable monitoring and AI safety. However, most existing analyses rely on topic- or task-specific prompts, which can substantially limit what can be observed. In this work, we study what LLMs generate from minimal, topic-neutral inputs and probe their near-unconstrained generative behavior. Despite the absence of explicit topics, model outputs cover a broad semantic space, and surprisingly, each model family exhibits strong and systematic topical preferences. GPT-OSS predominantly generates programming (27.1%) and mathematical content (24.6%), whereas Llama most frequently generates literary content (9.1%). DeepSeek often generates religious content, while Qwen frequently generates multiple-choice questions. Beyond topical preferences, we also observe differences in content specialization and depth: GPT-OSS often generates more technically advanced content (e.g., dynamic programming) compared with other models (e.g., basic Python). Furthermore, we find that the near-unconstrained generation often degenerates into repetitive phrases, revealing interesting behaviors unique to each model family. For instance, degenerate outputs from Llama include multiple URLs pointing to personal Facebook and Instagram accounts. We release the complete dataset of 256,000 samples from 16 LLMs, along with a reproducible codebase.

Figures (19)

• Figure 1: LLM's top-of-mind behaviors. UMAP visualizations of model outputs generated by the GPT-OSS, DeepSeek, Llama, and Qwen model families. Figure (a) visualizes outputs from all four model families, with each dot representing a generated output and class labels positioned at the centroid of their clusters. Except for ‘multiple-choice exam questions’ and ‘algorithms,’ cluster labels correspond to the 13 most frequent categories in our dataset. The two labels are included to better highlight clusters that are particularly prominent in Qwen and GPT-OSS. Figure (b) illustrates an individual model family, with the black dotted line indicating the convex hull of the high-density region. Figure (c) shows the top five categories within each model family and their corresponding percentages. The color scheme is consistent across all figures. Despite the use of topic-neutral seed prompts, the semantic distributions of model outputs show a broad and diverse range of topics, and each model family exhibits distinctive distributional patterns. Methodological details are provided in Section \ref{['sec:methods']}, and a high-quality interactive figure with fine-grained labels is available in Supplementary Material.
• Figure 2: Examples of model outputs and their category labels. We find that most generated texts appear coherent and meaningful, and that the LLM-based labeler produces sensible label annotations. For instance, the GPT-OSS sample describes an attempt to solve a problem that appears to be from the USA Computing Olympiad using a dynamic programming algorithm and is categorized as programming. The DeepSeek sample references Bible verses from Matthew and Hebrews and is categorized as religion. Llama generates a segment of fictional narrative, and Qwen outlines an approach to developing critical thinking; these outputs are accordingly categorized as literature and education, respectively.
• Figure 3: Distribution of subcategories for programming and mathematics. Numbers in each cell indicate the percentage of outputs that fall into a given subcategory among all outputs generated by the corresponding model family within a specific category. For each category, we select the nine subcategories with the highest average percentages and order them based on their size. Each model family demonstrates distinct specialization across subcategories.
• Figure 4: Distribution of depth levels in (left) programming and (right) mathematics. GPT-OSS mainly produces advanced or expert-level content in both programming and mathematics, while Llama and Qwen generate mostly basic or intermediate-level text.
• Figure 5: Distinct degenerate text behavior across model families. The left figure shows how likely model outputs include degenerate text, the middle shows the start index of degenerate text within the generated sequence, and the right figure shows the average length of repeated phrases. We observe substantial variation across model families in the frequency of degenerate text, the position at which it begins, and the length of repetitive phrases. In particular, degenerate text occurs in only 1.9% of DeepSeek generations, while it appears in 10.3% of Qwen generations, representing a 5.4-fold increase.

Theorems & Definitions (3)

• Remark 2.1: The impact of chat templates
• Remark 3.1
• Remark 3.2: Topical preference and task competence