TableTale: Reviving the Narrative Interplay Between Data Tables and Text in Scientific Papers

TL;DR

TableTale is presented, an augmented reading interface that enriches text with data tables at multiple granularities, including paragraphs, sentences, and mentions, and progressively renders cascade visual cues on text and tables that unfold as readers move through the text.

Abstract

Data tables play a central role in scientific papers. However, their meaning is often co-constructed with surrounding text through narrative interplay, making comprehension cognitively demanding for readers. In this work, we explore how interfaces can better support this reading process. We conducted a formative study that revealed key characteristics of text-table narrative interplay, including linking mechanisms, multi-granularity alignments, and mention typologies, as well as a layered framework of readers' intents. Informed by these insights, we present TableTale, an augmented reading interface that enriches text with data tables at multiple granularities, including paragraphs, sentences, and mentions. TableTale automatically constructs a document-level linking schema within the paper and progressively renders cascade visual cues on text and tables that unfold as readers move through the text. A within-subject study with 24 participants showed that TableTale reduced cognitive workload and improved reading efficiency, demonstrating its potential to enhance paper reading and inform future reading interface design.

Figures (6)

• Figure 1: Example of narrative interplay between text and data table. An excerpt from Bender et al. bender2021dangers shows how text and the related table jointly construct meaning. The narrative references the table through (1) semantic links using model names, such as MegatronLM and GPT-3; (2) numeric links requiring readers to reconcile textual values (e.g., 1.6T parameters) with scientific notation in the table (e.g., 1.57E+12); and (3) structural links that reuse descriptors such as parameters and data size.
• Figure 2: Alignment pattern between tables and text. Table-side granularity ranges from full tables to regions, rows, columns, and cells, while text-side granularity spans paragraphs, sentences, and mentions. The linking mechanisms include semantic, numeric, and structural connections that align units across the two modalities.
• Figure 3: TableTale interface with Progressive Cascade Activation. (a) A sidebar cue indicates paragraphs containing table references; activating it anchors the corresponding table next to the paragraph even if the table is off-screen. (b) Hovering over a sentence highlights only the relevant cells, rows, columns, or regions in the anchored table. (c) Clicking a sentence reveals its mentions, and hovering over a mention (e.g., "12.5%") further highlights the exact evidence cells (e.g., 85.1 and 72.6).
• Figure 4: Pipeline of TableTale. The system processes an input paper through four stages: (1) parsing and structuring, (2) paragraph-table matching, (3) fine-grained text-to-table alignment via multi-agent collaboration, and (4) localization and interactive rendering.
• Figure 5: Overview of our multi-granular text–table alignment framework. (A) Linking schema data model that organizes paragraphs, sentences, mentions, and their alignment targets. (B) Bottom-up alignment procedure illustrated with an example sentence, showing how fine-grained mention links are progressively merged into sentence-level table regions.