Toward a Machine Bertin: Why Visualization Needs Design Principles for Machine Cognition

TL;DR

The paper questions whether six decades of human-centered visualization knowledge transfers to machine cognition, particularly in vision-language models that parse charts. It analyzes benchmark and perceptual-comparison evidence to show that machines process visualizations via patch-based encoding, OCR-reliant text extraction, and sequential attention, leading to qualitative differences from human perception and distinct failure modes. It argues against the prevailing strategy of bypassing vision and proposes a research agenda for machine-oriented visualization, culminating in the notion of a machine Bertin to establish empirical foundations for machine-specific design principles. This work aims to expand visualization theory to machine audiences and guide future empirical work on how visual representations could most effectively support machine reasoning in AI pipelines.

Abstract

Visualization's design knowledge-effectiveness rankings, encoding guidelines, color models, preattentive processing rules -- derives from six decades of psychophysical studies of human vision. Yet vision-language models (VLMs) increasingly consume chart images in automated analysis pipelines, and a growing body of benchmark evidence indicates that this human-centered knowledge base does not straightforwardly transfer to machine audiences. Machines exhibit different encoding performance patterns, process images through patch-based tokenization rather than holistic perception, and fail on design patterns that pose no difficulty for humans-while occasionally succeeding where humans struggle. Current approaches address this gap primarily by bypassing vision entirely, converting charts to data tables or structured text. We argue that this response forecloses a more fundamental question: what visual representations would actually serve machine cognition well? This paper makes the case that the visualization field needs to investigate machine-oriented visual design as a distinct research problem. We synthesize evidence from VLM benchmarks, visual reasoning research, and visualization literacy studies to show that the human-machine perceptual divergence is qualitative, not merely quantitative, and critically examine the prevailing bypassing approach. We propose a conceptual distinction between human-oriented and machine-oriented visualization-not as an engineering architecture but as a recognition that different audiences may require fundamentally different design foundations-and outline a research agenda for developing the empirical foundations the field currently lacks: the beginnings of a "machine Bertin" to complement the human-centered knowledge the field already possesses.

Figures (1)

• Figure 1: The conceptual distinction this paper draws. Visualization's design knowledge for human audiences (left) is grounded in six decades of perceptual science---a well-established body of research. Benchmark evidence (center) indicates that this knowledge does not necessarily transfer to machine audiences, whose visual processing differs in mechanism, encoding effectiveness, and failure modes. The corresponding design knowledge for machine-oriented visualization (right) does not yet exist. This paper argues that developing it is a research direction worth pursuing.