TikZilla: Scaling Text-to-TikZ with High-Quality Data and Reinforcement Learning

TL;DR

This work constructs DaTikZ-V4, a dataset more than four times larger and substantially higher in quality than DaTikZ-V3, enriched with LLM-generated figure descriptions, and trains TikZilla, a family of small open-source Qwen models with a two-stage pipeline of SFT followed by reinforcement learning (RL).

Abstract

Large language models (LLMs) are increasingly used to assist scientists across diverse workflows. A key challenge is generating high-quality figures from textual descriptions, often represented as TikZ programs that can be rendered as scientific images. Prior research has proposed a variety of datasets and modeling approaches for this task. However, existing datasets for Text-to-TikZ are too small and noisy to capture the complexity of TikZ, causing mismatches between text and rendered figures. Moreover, prior approaches rely solely on supervised fine-tuning (SFT), which does not expose the model to the rendered semantics of the figure, often resulting in errors such as looping, irrelevant content, and incorrect spatial relations. To address these issues, we construct DaTikZ-V4, a dataset more than four times larger and substantially higher in quality than DaTikZ-V3, enriched with LLM-generated figure descriptions. Using this dataset, we train TikZilla, a family of small open-source Qwen models (3B and 8B) with a two-stage pipeline of SFT followed by reinforcement learning (RL). For RL, we leverage an image encoder trained via inverse graphics to provide semantically faithful reward signals. Extensive human evaluations with over 1,000 judgments show that TikZilla improves by 1.5-2 points over its base models on a 5-point scale, surpasses GPT-4o by 0.5 points, and matches GPT-5 in the image-based evaluation, while operating at much smaller model sizes. Code, data, and models will be made available.

Figures (15)

• Figure 1: Left: human evaluation of caption quality by structural elements and usefulness ratings. Right: BLEU-4, ROUGE-L, STS, and average length for raw captions, VLM-generated descriptions, and human-written descriptions (using other human descriptions as references).
• Figure 2: Overview of the data preprocessing workflow. We start by sourcing TikZ graphics programs primarily from arXiv, GitHub, TeX SE, as well as synthetic data. Next, rule-based filtering techniques are applied, and the TikZ code is rendered. Uncompilable code undergoes an iterative debugging process using LLMs alongside the error messages to attempt error correction. Finally, all compilable code images are described using VLMs.
• Figure 3: Overview of our post-SFT optimization steps. We first fully finetune DeTikZify-V2 consisting of an image encoder (IE), linear layer (LL) and LLM decoder (DEC) on our larger DaTikZ-V4 where we then use its enhanced IE to further finetune our LLMs based on the semantic similarity of the embeddings between ground truth and rendered image in an online RL setting using GRPO. The IE is kept frozen during RL optimization to mitigate reward hacking.
• Figure 4: Average Likert-scale ratings (1–5, higher is better) with 95% confidence intervals for eight LLMs, evaluated under two settings: (i) alignment with textual descriptions and (ii) alignment with ground-truth images. Combined scores are shown as the average of both settings.
• Figure 5: SFT on Qwen2.5-3B with different dataset scales (75%, 50%, 25%, 12.5%, and 6.25%).