PICABench: How Far Are We from Physically Realistic Image Editing?

TL;DR

This work addresses the gap between semantic image editing and physically realistic edits. It introduces PICABench, a fine-grained benchmark spanning optics, mechanics, and state transitions, paired with PICAEval, a region-grounded QA-based evaluation to robustly assess physical plausibility. It also proposes PICA-100K, a synthetic video-derived dataset, and demonstrates that video-based supervision improves physical realism when finetuning editors, while real-video data may be less effective. Overall, the study provides a practical framework and resources to push image editing research from semantic accuracy toward physically grounded realism.

Abstract

Image editing has achieved remarkable progress recently. Modern editing models could already follow complex instructions to manipulate the original content. However, beyond completing the editing instructions, the accompanying physical effects are the key to the generation realism. For example, removing an object should also remove its shadow, reflections, and interactions with nearby objects. Unfortunately, existing models and benchmarks mainly focus on instruction completion but overlook these physical effects. So, at this moment, how far are we from physically realistic image editing? To answer this, we introduce PICABench, which systematically evaluates physical realism across eight sub-dimension (spanning optics, mechanics, and state transitions) for most of the common editing operations (add, remove, attribute change, etc.). We further propose the PICAEval, a reliable evaluation protocol that uses VLM-as-a-judge with per-case, region-level human annotations and questions. Beyond benchmarking, we also explore effective solutions by learning physics from videos and construct a training dataset PICA-100K. After evaluating most of the mainstream models, we observe that physical realism remains a challenging problem with large rooms to explore. We hope that our benchmark and proposed solutions can serve as a foundation for future work moving from naive content editing toward physically consistent realism.

Figures (19)

• Figure 1: Challenging cases from PICABench. Despite providing instruction-aligned outputs, current SoTA models still struggle with generating physically realistic edits, resulting in unharmonized lighting, deformation, or state transitions with common editing operations.
• Figure 2: Statistics Analysis of PICABench. PICABench is a comprehensive benchmark designed to evaluate physical realism of image editing models across eight sub-dimentinons. Fig. \ref{['fig:benchmark_distribution']}(a) shows distribution of QA pairs. Fig. \ref{['fig:benchmark_distribution']}(b) presents words length distribution of editing instruction across three levels of prompt. Fig. \ref{['fig:benchmark_distribution']}(c) provides a perspective on overall composition of PICABench.
• Figure 3: Overview of PICABench. We present illustrative examples from eight sub-dimensions. Key regions are annotated to help reduce hallucination for VLMs.
• Figure 4: Overall pipeline for benchmarks construction and evaluation. (a–b) We enrich a physics-specific keyword set and retrieve diverse candidate images. (c–d) Human-written editing instructions are expanded into three levels of complexity using GPT-5. (e) Annotators mark physics-critical regions. (f) Spatially grounded yes/no questions are generated to evaluate physical plausibility. (g) During evaluation, VLMs answer each question with reference to the edited region.
• Figure 5: PICA-100K construction pipeline. We first curate structured prompts for scene and subject composition, refined by GPT-5 and rendered using FLUX.1-Krea-dev for text-to-image generation. Motion-based edit instructions are created via GPT-5 and applied using Wan2.2-14B to synthesize short videos depicting physical transformations.