AesTest: Measuring Aesthetic Intelligence from Perception to Production

TL;DR

This work introduces AesTest, a unified benchmark to assess aesthetic intelligence in multimodal large language models across perception, appreciation, creation, and photography. By organizing ten sub-tasks from diverse data sources into a consistent multiple-choice format, it enables scalable measurement of both understanding and production of aesthetics. Experiments reveal that while general MLLMs excel at perceptual tasks, production-oriented tasks such as creation and photography remain challenging, with notable gaps between model performance and professional human capabilities. The authors provide public release of AesTest to catalyze cognitively grounded progress in vision-language systems that can reason about and generate aesthetically informed content.

Abstract

Perceiving and producing aesthetic judgments is a fundamental yet underexplored capability for multimodal large language models (MLLMs). However, existing benchmarks for image aesthetic assessment (IAA) are narrow in perception scope or lack the diversity needed to evaluate systematic aesthetic production. To address this gap, we introduce AesTest, a comprehensive benchmark for multimodal aesthetic perception and production, distinguished by the following features: 1) It consists of curated multiple-choice questions spanning ten tasks, covering perception, appreciation, creation, and photography. These tasks are grounded in psychological theories of generative learning. 2) It integrates data from diverse sources, including professional editing workflows, photographic composition tutorials, and crowdsourced preferences. It ensures coverage of both expert-level principles and real-world variation. 3) It supports various aesthetic query types, such as attribute-based analysis, emotional resonance, compositional choice, and stylistic reasoning. We evaluate both instruction-tuned IAA MLLMs and general MLLMs on AesTest, revealing significant challenges in building aesthetic intelligence. We will publicly release AesTest to support future research in this area.

Figures (9)

• Figure 1: The overall architecture of our AesTest. It has four tasks: Perception, Appreciation, Creation, and Photography. Perception assesses image aesthetics (e.g., score, content, style, emotion). Appreciation elicits connoisseur-level critique. Creation tests a model’s ability to improve aesthetics via curation and retouching. Photography probes knowledge of principles for capturing impactful images.
• Figure 2: Dataset statistics. (a) Task distribution, (b) Dataset distribution, where Self-Collected refers to images we curated specifically for AesTest.
• Figure 3: Examples of each task.
• Figure 4: An exemplar of aesthetic framing choice Q&A construction. The orange box marks the pre-expansion framing, and the red box marks the distractor framing.
• Figure 5: Sub-task performance of different methods.