Contrast Sensitivity in Multimodal Large Language Models: A Psychophysics-Inspired Evaluation

TL;DR

The paper tackles how multimodal large language models perceive low-level visual structure by adapting human psychophysics to MLMMs, treating models as end-to-end observers and measuring their contrast sensitivity across spatial frequencies using bandpass-filtered noise. It introduces a prompt-aware, stimulus-driven method that fits Weibull psychometric functions to detectability data and derives CSFs without inspecting internal representations. Key findings show substantial diversity in CSF shapes and scales across models, with some echoing human bandpass patterns and others deviating significantly, and reveal that CSFs are sensitive to prompt phrasing while still predicting downstream robustness under frequency-based perturbations. The work positions CSF profiling as a scalable diagnostic tool for perceptual evaluation, with practical implications for robustness, prompt design, and future perceptual-alignment frameworks in multimodal systems, and it discusses limitations and avenues for naturalistic stimuli and broader benchmarking.

Abstract

Understanding how Multimodal Large Language Models (MLLMs) process low-level visual features is critical for evaluating their perceptual abilities and has not been systematically characterized. Inspired by human psychophysics, we introduce a behavioural method for estimating the Contrast Sensitivity Function (CSF) in MLLMs by treating them as end-to-end observers. Models are queried with structured prompts while viewing noise-based stimuli filtered at specific spatial frequencies. Psychometric functions are derived from the binary verbal responses, and contrast thresholds (and CSFs) are obtained without relying on internal activations or classifier-based proxies. Our results reveal that some models resemble human CSFs in shape or scale, but none capture both. We also find that CSF estimates are highly sensitive to prompt phrasing, indicating limited linguistic robustness. Finally, we show that CSFs predict model performance under frequency-filtered and adversarial conditions. These findings highlight systematic differences in frequency tuning across MLLMs and establish CSF estimation as a scalable diagnostic tool for multimodal perception.

Figures (7)

• Figure 1: Method summary: Overview of the proposed psychophysics-inspired method for estimating contrast sensitivity functions (CSFs) in multimodal large language models (MLLMs). Models are presented with bandpass-filtered noise images at varying spatial frequencies and contrast levels, accompanied by natural language prompts. The binary (“yes/no”) responses are used to build psychometric functions, from which contrast detection thresholds are extracted. CSFs are computed as the inverse of these thresholds and analyzed across models and prompts.
• Figure 2: Psychometric functions fitted: Example of the psychometric functions fitted with the model's experimental data. Blue points correspond with the percentage of affirmative model answers for each contrast. The orange line is the fitted psychometric function, independent for each frequency. Finally, the red lines mark the 50% detection point and the associated contrast threshold. In this case, the results are from the model Qwen2.5VL-7B with the prompt “$<$image$>$ Is there an observable pattern in the image? Respond just yes or no”.
• Figure 3: Average contrast sensitivity functions (CSFs) for all tested models: Each curve represents the average CSF across 25 prompt variants per spatial frequency. Note that we used log-log scales for better visualization. The human CSF (of the Standard Spatial Observer Watson02) is included for reference as the dark blue line.
• Figure 4: Prompt sensitivity and consistency across models Left: average entropy of model responses across 25 prompt formulations, indicating how much a model’s detection behaviour varies with phrasing (higher values = less stable). Right: prompt consistency, computed as the mean pairwise agreement between prompt responses per contrast–frequency condition (higher values = more stable). These metrics quantify how robust each model’s CSF estimation is to natural language variation in the query.
• Figure 5: Frequency-specific adversarial and filtered images: Example of the adversarial (top) and filtered (bottom) effect on an image depending on the filtered frequency band (columns) and the induced variance (rows).