The Promise of RL for Autoregressive Image Editing

TL;DR

This paper addresses text-guided image editing by evaluating three learning paradigms—supervised fine-tuning, reinforcement learning post-training, and chain-of-thought reasoning—within a unified autoregressive multimodal model (Emu3) and introduces EARL, an RL-enhanced editing framework. The key finding is that RL post-training, guided by a strong multimodal verifier, delivers the most reliable and broad improvements across simple and complex edits, achieving state-of-the-art or competitive results with substantially less training data than diffusion-based baselines. Chain-of-thought reasoning does not consistently help in this setting and can even degrade performance, while complex edits should be introduced during RL rather than SFT to maximize gains. Overall, EARL demonstrates data-efficient, autoregressive editing that rivals diffusion-based methods and highlights the critical role of training stage and verifier quality in RL for multimodal editing.

Abstract

While image generation techniques are now capable of producing high-quality images that respect prompts which span multiple sentences, the task of text-guided image editing remains a challenge. Even edit requests that consist of only a few words often fail to be executed correctly. We explore three strategies to enhance performance on a wide range of image editing tasks: supervised fine-tuning (SFT), reinforcement learning (RL), and Chain-of-Thought (CoT) reasoning. In order to study all these components in one consistent framework, we adopt an autoregressive multimodal model that processes textual and visual tokens in a unified manner. We find RL combined with a large multi-modal LLM verifier to be the most effective of these strategies. As a result, we release EARL: Editing with Autoregression and RL, a strong RL-based image editing model that performs competitively on a diverse range of edits compared to strong baselines, despite using much less training data. Thus, EARL pushes the frontier of autoregressive multimodal models on image editing. We release our code, training data, and trained models at https://github.com/mair-lab/EARL.

Figures (8)

• Figure 1: Qualitative comparison between SFT-only and EARL across diverse editing instructions. EARL extends the SFT model by leveraging reinforcement learning to better align image edits with natural language prompts. While both models handle simple edits reasonably well, EARL exhibits clear improvements in precise editing on simple as well as complex edit instructions. Simple edit instructions are shown in blue, and complex edit instructions are shown in pink.
• Figure 2: Autoregressive Image Editing Approaches. In supervised fine-tuning (SFT), we train an autoregressive model based on the standard image editing setup: triplets of source image, edit instruction, and target image. In SFT with reasoning, the model is supervised to generate chain-of-thought (CoT) reasoning traces before generating the final edited image. Finally, we study reinforcement learning (RL) training of the SFT checkpoint, using edit quality verifiers as reward signals.
• Figure 3: Example of step-by-step reasoning generated by Qwen2.5-VL-72B using standard editing data (input image, edit instruction, target image, and bounding box).
• Figure 4: (a) Training curves showing the reward progression, with different aspects of reward. and (b) VIEScore on OmniEdit increases with RL training iterations
• Figure 5: Example reward scores for various types of image edits using Qwen2.5-VL-72B. Higher scores reflect better alignment with the intended edit prompt.