InsigHTable: Insight-driven Hierarchical Table Visualization with Reinforcement Learning

TL;DR

InsigHTable tackles the challenge of embedding rich data insights within hierarchical tabular visualizations by modeling the construction process as a two-stage deep reinforcement learning problem. It introduces an insight taxonomy for hierarchical data, a Markov Decision Process with transformation and selection actions, and an auxiliary curiosity-driven reward to mitigate sparse rewards during visualization embedding. The system combines a GCN-based heading encoder with Bi-LSTM content modeling and an Actor-Critic agent that jointly optimizes table transformations and visualization regions, guided by both extrinsic and intrinsic rewards. Evaluation on real datasets via case studies and extensive experiments demonstrates effective insight discovery, robust performance across diverse table structures, and advantages over heuristic baselines in producing integrated, interpretable visualizations. The work advances scalable, mixed-initiative visualization authoring for complex hierarchical data with practical implications for analysts and decision-makers.

Abstract

Embedding visual representations within original hierarchical tables can mitigate additional cognitive load stemming from the division of users' attention. The created hierarchical table visualizations can help users understand and explore complex data with multi-level attributes. However, because of many options available for transforming hierarchical tables and selecting subsets for embedding, the design space of hierarchical table visualizations becomes vast, and the construction process turns out to be tedious, hindering users from constructing hierarchical table visualizations with many data insights efficiently. We propose InsigHTable, a mixed-initiative and insight-driven hierarchical table transformation and visualization system. We first define data insights within hierarchical tables, which consider the hierarchical structure in the table headers. Since hierarchical table visualization construction is a sequential decision-making process, InsigHTable integrates a deep reinforcement learning framework incorporating an auxiliary rewards mechanism. This mechanism addresses the challenge of sparse rewards in constructing hierarchical table visualizations. Within the deep reinforcement learning framework, the agent continuously optimizes its decision-making process to create hierarchical table visualizations to uncover more insights by collaborating with analysts. We demonstrate the usability and effectiveness of InsigHTable through two case studies and sets of experiments. The results validate the effectiveness of the deep reinforcement learning framework and show that InsigHTable can facilitate users to construct hierarchical table visualizations and understand underlying data insights.

Figures (8)

• Figure 1: (a) is an example of a hierarchical table, while (b) provides annotations for the key constituents of this hierarchical table, encompassing cells, blocks, column headings, and row headings. (c) displays the corresponding flat table counterpart of (a), where the contents of the hierarchical table are exclusively positioned within the final column (value). (d) shows a hierarchical table visualization. This visualization involves the incorporation of summary visualizations within a contiguous block, alongside unit visualizations within individual cells.
• Figure 2: Insight classifications for hierarchical tabular data. The left part shows the single-block insights, encompassing point insights, shape insights, and compound insights. The right part shows the multiple-block insights, which is defined through both name-based recommendation and topology-based recommendation mechanisms. The selected entries within the table headings are highlighted in green.
• Figure 3: The actions conducted by the agents of the reinforcement learning model are divided into two groups, the transformation action of the hierarchical table, and the selection action of data items to be visualized.
• Figure 4: The construction process of InsigHTable involves two stages: transformation and visualization. These stages perform different operations on the hierarchical table. During the transformation stage, the table headers and content are updated accordingly. In the visualization stage, data blocks within the hierarchical table are identified, and visualization results are inserted. $state_0$ represents the input table, while $state_m$ denotes the transformation results. $state_{m+n}$ corresponds to the visualization results. The model's structure remains the same in both stages, except for the action output by the agent module, which is indicated by an ellipse with a dotted border to highlight their differences. The bottom part of the model structure consists of the agent module and the curiosity module.
• Figure 5: The hierarchical table visualization is based on a dataset of console sales and is created using the InsigHTable prototype system. This data is about sales data for Nintendo, Sony, and Microsoft, and encompasses generated visualizations that are embedded in the table content. The InsigHTable prototype system consists of four panels: (A) table visualization panel; (B) insight list panel; (C) alternative insight panel; and (D) visualization configuration panel.