Understanding Counting Mechanisms in Large Language and Vision-Language Models

TL;DR

The paper examines how large language and vision-language models internally represent counting and compute numerical outputs. It introduces CountScope, a causal, patch-based probe, and applies causal mediation analysis to dissect where counting information resides and how a latent internal counter evolves across items and layers. Key findings include progressive, layerwise emergence of numerical representations, a transferable internal counter that updates with each item, and separator tokens acting as strong structural shortcuts; a max latent count effect further shapes final outputs. The work reveals cross-modal similarities and modality-specific differences in counting, with implications for designing models with more robust numerical abstraction and memory. These insights advance mechanistic interpretability of numeracy in AI systems and offer practical targets for improving counting reliability in future LLMs and LVLMs.

Abstract

This paper examines how large language models (LLMs) and large vision-language models (LVLMs) represent and compute numerical information in counting tasks. We use controlled experiments with repeated textual and visual items and analyze model behavior through causal mediation and activation patching. To this end, we design a specialized tool, CountScope, for mechanistic interpretability of numerical content. Results show that individual tokens or visual features encode latent positional count information that can be extracted and transferred across contexts. Layerwise analyses reveal a progressive emergence of numerical representations, with lower layers encoding small counts and higher layers representing larger ones. We identify an internal counter mechanism that updates with each item, stored mainly in the final token or region and transferable between contexts. In LVLMs, numerical information also appears in visual embeddings, shifting between background and foreground regions depending on spatial composition. Models rely on structural cues such as separators in text, which act as shortcuts for tracking item counts and influence the accuracy of numerical predictions. Overall, counting emerges as a structured, layerwise process in LLMs and follows the same general pattern in LVLMs, shaped by the properties of the vision encoder.

Figures (16)

• Figure 1: A minimal graphical abstract illustrating how CountScope causally reveals a fundamental mechanism (such as the latent counter shown here) in language models.
• Figure 2: Representational behavior of embeddings in a selected layer of the LLM. Left: PCA projection of embeddings corresponding to items at different list positions. Right: Cross-task (different item types) cosine similarity of item embeddings, averaged over the dataset.
• Figure 3: Ground-truth (total count) probability of visual objects decoded by CountScope. Besides the last object, nearby objects also contribute to the prediction, indicating distributed count information across adjacent regions.
• Figure 4: Per-item latent count encoding, decoded by CountScope. Each heatmap shows the probability of decoding numbers (1–9) across sequence positions (1–9) for (a) textual monotypic, (b) textual polytypic with unique items, and (c) visual monotypic tasks. Textual tasks use the question-first configuration.
• Figure 5: Type-specific counter behavior revealed by CountScope. Each bar shows the average probability of digits, averaged over 500 different configurations and 9 digits. Gen indicates total number of items, while Sp denotes the specific counter. Sp-A refers to the total number of separated items of a type, and Sp-L indicates the count of the last group of that type. Left: textual task; middle: visual task; right: textual task with repeated groups of one type.