QualiRAG: Retrieval-Augmented Generation for Visual Quality Understanding

TL;DR

QualiRAG tackles visual quality understanding by reframing it as a retrieval-grounded reasoning task that leverages latent multimodal knowledge without task-specific training. It decomposes queries with a Granularity-Aware Query Formulation, augments evidence via four complementary knowledge sources, retrieves relevant material through a relevance-aware selector, and grounds final answers in both the visual input and retrieved content. The approach yields strong, backbone-agnostic performance on image and video quality understanding benchmarks and competitive results on quality comparison, validating the effectiveness of a training-free RAG paradigm for interpretable quality assessment. Its modular design allows dynamic extension with domain-specific detectors, offering scalable applicability across real-world multimedia pipelines.

Abstract

Visual quality assessment (VQA) is increasingly shifting from scalar score prediction toward interpretable quality understanding -- a paradigm that demands \textit{fine-grained spatiotemporal perception} and \textit{auxiliary contextual information}. Current approaches rely on supervised fine-tuning or reinforcement learning on curated instruction datasets, which involve labor-intensive annotation and are prone to dataset-specific biases. To address these challenges, we propose \textbf{QualiRAG}, a \textit{training-free} \textbf{R}etrieval-\textbf{A}ugmented \textbf{G}eneration \textbf{(RAG)} framework that systematically leverages the latent perceptual knowledge of large multimodal models (LMMs) for visual quality perception. Unlike conventional RAG that retrieves from static corpora, QualiRAG dynamically generates auxiliary knowledge by decomposing questions into structured requests and constructing four complementary knowledge sources: \textit{visual metadata}, \textit{subject localization}, \textit{global quality summaries}, and \textit{local quality descriptions}, followed by relevance-aware retrieval for evidence-grounded reasoning. Extensive experiments show that QualiRAG achieves substantial improvements over open-source general-purpose LMMs and VQA-finetuned LMMs on visual quality understanding tasks, and delivers competitive performance on visual quality comparison tasks, demonstrating robust quality assessment capabilities without any task-specific training. The code will be publicly available at https://github.com/clh124/QualiRAG.

Figures (5)

• Figure 1: Comparison of training-based and training-free paradigms for visual quality understanding. Unlike SFT- and RL-based LMMs that require task-specific training with human annotations or reward signals, our RAG-based LMM performs inference-time retrieval and reasoning without any additional training, providing improved flexibility and scalability.
• Figure 2: The overall workflow of our QualiRAG. Given a visual input and a quality-related question, QualiRAG first decomposes the query into structured requests via a Query Organizer, then constructs complementary auxiliary knowledge sources through a Source Augmenter. A Source Selector retrieves question-relevant evidence from multiple knowledge databases, which is finally integrated with sampled visual inputs by the Answer Generator to produce evidence-grounded and interpretable quality reasoning.
• Figure 3: Performance of LLaVA-OneVision-7B, InternVL3-9B-Instruct, InternVL3.5-8B-Instruct, Qwen2.5-VL-7B-Instruct, and Qwen3-VL-8B-Instruct on the dev subset of Q-Bench-Video, before and after applying QualiRAG.
• Figure 4: Examples of image quality understanding on Q-Bench.
• Figure 5: Examples of video quality understanding on Q-Bench-Video.