AutoLegend: A User Feedback-Driven Adaptive Legend Generator for Visualizations

TL;DR

AutoLegend tackles the problem of missing or poorly designed legends in visualizations by defining a five‑dimensional design space for legends and implementing a user feedback‑driven, real‑time legend generator. The system extracts iconic symbols and mapping channels, searches a high‑dimensional legend space with a genetic algorithm, and scores candidates with a quality assessment model that adapts online via adversarial training guided by user edits. It demonstrates bidirectional legend–visualization interactions and supports both single‑ and multi‑channel legends, with a user study showing that preference‑adjusted legends outperform defaults and that the model can learn user tastes. The work advances practical legend design by enabling automatic, adaptive legend generation and human‑AI collaboration to improve readability, mapping clarity, and visualization retargeting in real time.

Abstract

We propose AutoLegend to generate interactive visualization legends using online learning with user feedback. AutoLegend accurately extracts symbols and channels from visualizations and then generates quality legends. AutoLegend enables a two-way interaction between legends and interactions, including highlighting, filtering, data retrieval, and retargeting. After analyzing visualization legends from IEEE VIS papers over the past 20 years, we summarized the design space and evaluation metrics for legend design in visualizations, particularly charts. The generation process consists of three interrelated components: a legend search agent, a feedback model, and an adversarial loss model. The search agent determines suitable legend solutions by exploring the design space and receives guidance from the feedback model through scalar scores. The feedback model is continuously updated by the adversarial loss model based on user input. The user study revealed that AutoLegend can learn users' preferences through legend editing.

Figures (10)

• Figure 1: Representative legends from IEEE VIS papers. (a) Text as symbol goodcase_a; (b) Text accompanying a symbol goodcase_b; (c) Text-embedded symbolgoodcase_c; (d) Continuous color legend goodcase_d_upgoodcase_d_down; (e) Connected symbol layout goodcase_e_upgoodcase_e_middle_down; (f) Non-uniform symbol layout goodcase_f; (g) Nested symbol layoutgoodcase_g; (h) Matrix form with color-discrete symbol goodcase_h; (i) Continuous matrix form goodcase_i; (j) Flattened form goodcase_j; (k) Paralleled form goodcase_k; (l) Data-encoded symbolgoodcase_l.
• Figure 2: The design space of a legend refers to the multi-dimensional space constituted by different design options of a legend. It includes five dimensions: the iconic symbol type of the legend, the mapping channel, the symbol layout, the text layout, and the layout of multiple channel legends.
• Figure 3: The workflow of AutoLegend begins with a visualization chart. After extracting representative markers and channels, the legend search agent conducts a solution search within the design space. The search process is guided by a quality assessment model that accepts evaluation metrics as input and outputs a scalar score for directing the legend search agent. The legend with the highest score is sent to the front-end interface, where users can interact and adjust its settings. The adjustments to the legend settings reflect the user's preferences, which are used to update the parameters of quality assessment model via real-time adversarial training.
• Figure 4: The process of legend parsing involves two main parts: shape extraction and color extraction. Simple to complex shape elements will be obtained by shape matching, rotation algorithms, and clustering analysis to derive representative graphics and their shape-related channels. Color extraction starts with color clustering, then sorting the colors and finally classifying them into continuous, sequential, or discrete categories.
• Figure 5: The quality assessment model and adversarial loss. The quality assessment model assesses the metrics of various legends and user preferences to compute a score. Conversely, the adversarial loss scrutinizes whether the partial order of the two components corresponds with the user's anticipations. Finally, the quality assessment model is back-propagated with the adversarial loss.