Fine-Tuned Large Language Model for Visualization System: A Study on Self-Regulated Learning in Education

TL;DR

This work tackles the challenge of making domain-specific visualization systems intelligent for education by introducing a three-alignment framework that tightly integrates fine-tuned LLMs with visualization and user interactions. It then operationalizes the framework in Tailor-Mind, an interactive SRL-oriented visualization system for AI beginners, supported by a large-scale tuning-data pipeline and a tailored SRL workflow. The authors demonstrate the approach with a comprehensive SRL pipeline, a 74,932-entry tuning dataset, and a Baichuan2-7B-chat-based fine-tuning regime, showing improvements in model performance and user experience across usage scenarios and a controlled user study. The results suggest that combining fine-tuned LLMs with structured visualization can meaningfully enhance self-regulated learning and provides a generalizable path toward domain-specific, interactive educational agents with potential applications beyond AI education.

Abstract

Large Language Models (LLMs) have shown great potential in intelligent visualization systems, especially for domain-specific applications. Integrating LLMs into visualization systems presents challenges, and we categorize these challenges into three alignments: domain problems with LLMs, visualization with LLMs, and interaction with LLMs. To achieve these alignments, we propose a framework and outline a workflow to guide the application of fine-tuned LLMs to enhance visual interactions for domain-specific tasks. These alignment challenges are critical in education because of the need for an intelligent visualization system to support beginners' self-regulated learning. Therefore, we apply the framework to education and introduce Tailor-Mind, an interactive visualization system designed to facilitate self-regulated learning for artificial intelligence beginners. Drawing on insights from a preliminary study, we identify self-regulated learning tasks and fine-tuning objectives to guide visualization design and tuning data construction. Our focus on aligning visualization with fine-tuned LLM makes Tailor-Mind more like a personalized tutor. Tailor-Mind also supports interactive recommendations to help beginners better achieve their learning goals. Model performance evaluations and user studies confirm that Tailor-Mind improves the self-regulated learning experience, effectively validating the proposed framework.

Figures (7)

• Figure 1: Framework of integrating fine-tuned LLM into visualization system. The black lines indicate relationships in traditional processes, and the red lines highlight connections that are introduced or altered by the involvement of fine-tuned LLM.
• Figure 2: Workflow for applying the framework. All three phases of the workflow are designed to achieve the alignment challenges.
• Figure 3: Results of student surveys. The left chart indicates the difficulties encountered by beginners in SRL. The right side demonstrates their need for a visualization system that supports intelligent aids to SRL.
• Figure 4: In applying workflow to SRL in education, we outline the process in three phases. Phase involves establishing a fundamental understanding of the SRL task (A1) and collecting data on artificial intelligence (A2). The design requirements (B) align with those outlined in Sec.\ref{['sec: requirements']} from which we derive the tuning tasks. Phase details the SRL pipeline sub-tasks and visualizations (C1), leading to the creation of fine-tuning data (C2). In phase , we enhance the fine-tuning effects and visualization interactions by integrating user feedback within the visualization system.
• Figure 5: Refined fine-tuning datasets with examples, where different highlights indicate various alignments. Data construction for tuning tasks (A for T1, B for T2, C and D for T3) has undergone one iteration.