UniPercept: Towards Unified Perceptual-Level Image Understanding across Aesthetics, Quality, Structure, and Texture

TL;DR

UniPercept defines a unified perceptual-level image understanding framework spanning Image Aesthetics Assessment, Image Quality Assessment, and Image Structure & Texture Assessment. It introduces UniPercept-Bench, a three-domain Domain–Category–Criterion benchmark with Visual Rating and Visual Question Answering tasks, and trains UniPercept via Domain-Adaptive Pre-Training plus Task-Aligned RL to achieve robust cross-domain generalization. Empirical results show UniPercept outperforms existing MLLMs on perceptual tasks and can serve as a plug-and-play perceptual reward model for text-to-image generation and as a perceptual metric for evaluation. The work provides a principled, scalable path toward human-aligned perceptual reasoning in multimodal models and lays groundwork for perceptual-level analysis of large-scale image datasets.

Abstract

Multimodal large language models (MLLMs) have achieved remarkable progress in visual understanding tasks such as visual grounding, segmentation, and captioning. However, their ability to perceive perceptual-level image features remains limited. In this work, we present UniPercept-Bench, a unified framework for perceptual-level image understanding across three key domains: Aesthetics, Quality, Structure and Texture. We establish a hierarchical definition system and construct large-scale datasets to evaluate perceptual-level image understanding. Based on this foundation, we develop a strong baseline UniPercept trained via Domain-Adaptive Pre-Training and Task-Aligned RL, enabling robust generalization across both Visual Rating (VR) and Visual Question Answering (VQA) tasks. UniPercept outperforms existing MLLMs on perceptual-level image understanding and can serve as a plug-and-play reward model for text-to-image generation. This work defines Perceptual-Level Image Understanding in the era of MLLMs and, through the introduction of a comprehensive benchmark together with a strong baseline, provides a solid foundation for advancing perceptual-level multimodal image understanding.

Figures (14)

• Figure 2: Semantic-level vs. Perceptual-level understanding.
• Figure 3: Representative QA examples in UniPercept-Bench. Questions follow a three-level hierarchy of Domain–Category–Criterion, defining perceptual scope, specific visual aspects, and fine-grained criteria for constructing diverse, perception-oriented VQA tasks.
• Figure 3: Performance comparison of different models on UniPercept-Bench-VQA (IQA).
• Figure 4: Constuction pipeline of UniPercept-Bench. A three-stage process initial QA generation, reject sampling, and human refinement to produce high-quality perceptual-level QA pairs across aesthetics, quality, structure, and texture.
• Figure 5: Distribution of UniPercept-Bench across (a) Domain, (b) Category, and (c) Criterion. Zoom in for best view.