Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Code Aesthetics with Agentic Reward Feedback

Bang Xiao, Lingjie Jiang, Shaohan Huang, Tengchao Lv, Yupan Huang, Xun Wu, Lei Cui, Furu Wei

TL;DR

The paper defines code aesthetics for visually oriented code and introduces AesCode-358K and the OpenDesign benchmark to evaluate aesthetics in Python plotting and webpage design. A triadic agentic reward framework (execution, static aesthetics, interactive aesthetics) combined with GRPO-AR is used to train AesCoder models, achieving state-of-the-art results among open-source systems and competitive performance with proprietary models. The approach demonstrates that multi-agent, visually grounded reward signals can align LLMs with both functionality and human-perceived aesthetics, improving executability and the quality of rendered interfaces. Overall, the work advances practical coding assistants by enabling more aesthetically coherent and interactive code outputs.

Abstract

Large Language Models (LLMs) have become valuable assistants for developers in code-related tasks. While LLMs excel at traditional programming tasks such as code generation and bug fixing, they struggle with visually-oriented coding tasks, often producing suboptimal aesthetics. In this paper, we introduce a new pipeline to enhance the aesthetic quality of LLM-generated code. We first construct AesCode-358K, a large-scale instruction-tuning dataset focused on code aesthetics. Next, we propose agentic reward feedback, a multi-agent system that evaluates executability, static aesthetics, and interactive aesthetics. Building on this, we develop GRPO-AR, which integrates these signals into the GRPO algorithm for joint optimization of functionality and code aesthetics. Finally, we develop OpenDesign, a benchmark for assessing code aesthetics. Experimental results show that combining supervised fine-tuning on AesCode-358K with reinforcement learning using agentic reward feedback significantly improves performance on OpenDesign and also enhances results on existing benchmarks such as PandasPlotBench. Notably, our AesCoder-4B surpasses GPT-4o and GPT-4.1, and achieves performance comparable to large open-source models with 480B-685B parameters, underscoring the effectiveness of our approach.

Code Aesthetics with Agentic Reward Feedback

TL;DR

The paper defines code aesthetics for visually oriented code and introduces AesCode-358K and the OpenDesign benchmark to evaluate aesthetics in Python plotting and webpage design. A triadic agentic reward framework (execution, static aesthetics, interactive aesthetics) combined with GRPO-AR is used to train AesCoder models, achieving state-of-the-art results among open-source systems and competitive performance with proprietary models. The approach demonstrates that multi-agent, visually grounded reward signals can align LLMs with both functionality and human-perceived aesthetics, improving executability and the quality of rendered interfaces. Overall, the work advances practical coding assistants by enabling more aesthetically coherent and interactive code outputs.

Abstract

Large Language Models (LLMs) have become valuable assistants for developers in code-related tasks. While LLMs excel at traditional programming tasks such as code generation and bug fixing, they struggle with visually-oriented coding tasks, often producing suboptimal aesthetics. In this paper, we introduce a new pipeline to enhance the aesthetic quality of LLM-generated code. We first construct AesCode-358K, a large-scale instruction-tuning dataset focused on code aesthetics. Next, we propose agentic reward feedback, a multi-agent system that evaluates executability, static aesthetics, and interactive aesthetics. Building on this, we develop GRPO-AR, which integrates these signals into the GRPO algorithm for joint optimization of functionality and code aesthetics. Finally, we develop OpenDesign, a benchmark for assessing code aesthetics. Experimental results show that combining supervised fine-tuning on AesCode-358K with reinforcement learning using agentic reward feedback significantly improves performance on OpenDesign and also enhances results on existing benchmarks such as PandasPlotBench. Notably, our AesCoder-4B surpasses GPT-4o and GPT-4.1, and achieves performance comparable to large open-source models with 480B-685B parameters, underscoring the effectiveness of our approach.
Paper Structure (54 sections, 9 equations, 6 figures, 5 tables)

This paper contains 54 sections, 9 equations, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Aesthetics of AI-Generated Contents.
  4. Reward Systems in Reinforcement Learning.
  5. The AesCode-358K Dataset
  6. Python-Based Plot Data Construction
  7. Webpage Design Data Construction
  8. Agentic Reward Framework
  9. Execution Agent
  10. Static Aesthetics Agent
  11. Interactive Aesthetics Agent
  12. Agent Planning.
  13. Agent Interacting and Scoring.
  14. Discussions.
  15. Reward Aggregation
  16. ...and 39 more sections

Figures (6)

  • Figure 1: Performance comparison of different models on the OpenDesign benchmark. Left: static score evaluation. Right: interactive score evaluation.
  • Figure 2: Overview of the AesCoder pipeline, which integrates data construction, model training, and a weighted scoring mechanism. GRPO-AR coordinates performing GRPO with three specialized reward agents—Execution, Static Aesthetics, and Interactive Aesthetics—for comprehensive reward feedback.
  • Figure 3: Overall comparison and alignment between OpenDesign and human evaluators.
  • Figure 4: Case study comparing AesCoder-4B and baseline models on OpenDesign. The categories from top to bottom are: General Website, Data Visualization, 3D Design, Game Dev, UI Component.
  • Figure 5: Visualization of instruction embeddings before and after filtering.
  • ...and 1 more figures