Bridging Cognitive Gap: Hierarchical Description Learning for Artistic Image Aesthetics Assessment

TL;DR

This work tackles the challenge of quantitatively evaluating artistic image aesthetics by bridging perception, cognition, and emotion through hierarchical textual descriptions. It introduces the Refined Aesthetic Description (RAD) dataset and the ArtQuant framework, which couples multi-level aesthetic description generation with auxiliary description learning and a Score-Based Distribution Estimation strategy to predict continuous aesthetic scores. An information-theoretic analysis provides guarantees on how description quality and sufficiency bound score prediction uncertainty, guiding model design and training. Empirically, ArtQuant achieves state-of-the-art results across multiple AIAA datasets while requiring only about 33% of conventional training epochs, demonstrating practical efficiency and effectiveness in narrowing the cognitive gap between human judgments and machine evaluation.

Abstract

The aesthetic quality assessment task is crucial for developing a human-aligned quantitative evaluation system for AIGC. However, its inherently complex nature, spanning visual perception, cognition, and emotion, poses fundamental challenges. Although aesthetic descriptions offer a viable representation of this complexity, two critical challenges persist: (1) data scarcity and imbalance: existing dataset overly focuses on visual perception and neglects deeper dimensions due to the expensive manual annotation; and (2) model fragmentation: current visual networks isolate aesthetic attributes with multi-branch encoder, while multimodal methods represented by contrastive learning struggle to effectively process long-form textual descriptions. To resolve challenge (1), we first present the Refined Aesthetic Description (RAD) dataset, a large-scale (70k), multi-dimensional structured dataset, generated via an iterative pipeline without heavy annotation costs and easy to scale. To address challenge (2), we propose ArtQuant, an aesthetics assessment framework for artistic images which not only couples isolated aesthetic dimensions through joint description generation, but also better models long-text semantics with the help of LLM decoders. Besides, theoretical analysis confirms this symbiosis: RAD's semantic adequacy (data) and generation paradigm (model) collectively minimize prediction entropy, providing mathematical grounding for the framework. Our approach achieves state-of-the-art performance on several datasets while requiring only 33% of conventional training epochs, narrowing the cognitive gap between artistic images and aesthetic judgment. We will release both code and dataset to support future research.

Figures (6)

• Figure 1: Top: Hierarchical cognitive mechanism of human artistic aesthetics, serving as our theoretical foundation. Bottom left: we employ an iterative data collection framework to optimize the training data distribution, ensuring that the generated aesthetic analysis aligns with human ratings. Bottom right: Dual-task learning (description generation + scoring prediction) enhances model's alignment with human judgement.
• Figure 2: Grey font indicates limitations in the existing dataset APDD jin2024apddv2. The green, blue, and orange fonts represent Perception, Cognition and Emotion respectively, demonstrating the fine-grained and multi-level nature of our methodology.
• Figure 3: (a) We leverages a scalable, iterative framework to generate hierarchical aesthetic descriptions, ensuring the descriptions align with human scoring. (b) By employing Multi-Task Aesthetic Training to decompose hierarchical aesthetic elements and integrating a high-precision Score-Based Distribution Estimation method, ArtQuant achieves superior alignment with human artistic aesthetic scoring.
• Figure 4: Assessment performance with different description sufficiency and generation capability, where more descriptions and generation capability make the model's assessment more consistent with human.
• Figure 5: Qualitative results on the APDD test set. The vertical axis represents the error between model predictions and human scores, while the horizontal axis shows artistic images of varying aesthetic quality. Our method achieves better alignment with human judgments than other models across across paintings of different qualities.

Theorems & Definitions (3)

• Theorem 1: Description-Score Dependency Bound
• Theorem 2: Conditional Independence Bound
• Theorem 3: Error Propagation Bound