Tiled Prompts: Overcoming Prompt Underspecification in Image and Video Super-Resolution

TL;DR

This work tackles prompt underspecification in high-resolution image and video super-resolution by replacing a single global caption with dense, tile-specific prompts. By extracting localized prompts for each latent tile via a vision-language model and performing per-tile conditioning, the approach forms a product of tile posteriors that provides accurate, region-specific guidance while mitigating misalignment and hallucination. Across real-world datasets for both SISR and VSR, tiled prompts yield consistent gains in perceptual quality and text alignment, with only modest runtime overhead. The framework formalizes prompt underspecification, demonstrates practical tile-level prompting, and shows substantial improvements in both image and video SR tasks, offering a unified, scalable solution for high-resolution restoration with semantic fidelity.

Abstract

Text-conditioned diffusion models have advanced image and video super-resolution by using prompts as semantic priors, but modern super-resolution pipelines typically rely on latent tiling to scale to high resolutions, where a single global caption causes prompt underspecification. A coarse global prompt often misses localized details (prompt sparsity) and provides locally irrelevant guidance (prompt misguidance) that can be amplified by classifier-free guidance. We propose Tiled Prompts, a unified framework for image and video super-resolution that generates a tile-specific prompt for each latent tile and performs super-resolution under locally text-conditioned posteriors, providing high-information guidance that resolves prompt underspecification with minimal overhead. Experiments on high resolution real-world images and videos show consistent gains in perceptual quality and text alignment, while reducing hallucinations and tile-level artifacts relative to global-prompt baselines.

Figures (7)

• Figure 1: (Left) Relying on a single, global prompt for the latent tiling strategy during super-resolution causes prompt underspecification, leading to suboptimal reconstructions. (Right) Using tiled prompts solves ambiguity and provides accurate localized guidance needed to reconstruct high-quality details (e.g., text on signs are correctly generated only when their corresponding prompts are provided).
• Figure 2: (a) Baseline Methods: Conditioning super-resolution models solely on a single global text prompt demonstrates the problem of prompt underspecification. The global prompt, while broadly describing the image, proves insufficient to constrain the fine-grained super-resolution process. (b) Our Method (Tiled Prompts): Our framework leverages dense, context-aware tiled prompts for each region. This localized textual guidance provides precise semantic anchors, enabling the super-resolution model to produce significantly sharper, more coherent, and perceptually richer details.
• Figure 3: Our Tiled Prompts framework for VSR first divides the low-resolution video into a grid of spatio-temporal "tubes" or volumes, where each tube is a spatial patch that extends for the full duration of the video. A VLM then analyzes the local video content and generates a detailed text prompt that accurately describes the events within that patch. This localized prompt is then fed into the 3D VSR model to guide the reconstruction of that specific patch-volume, significantly improving textual guidance.
• Figure 4: Qualitative results for image super-resolution. (a,b) Input: The low-resolution input and a cropped tile to be upsampled. (c) SR with Global Prompt: The baseline result using only a single, coarse global prompt. As the text prompt does not provide sufficient guidance, super-resolution results are inaccurate. (d) SR with Tiled Prompt (Ours): Our method uses dense, localized textual guidance to generate significantly more accurate and semantically plausible high-frequency details consistent with the context.
• Figure 5: Qualitative results for video super-resolution. (a,b) Input: A low-resolution input frame and its cropped tile before SR. (c) SR with Global Prompt: Using only a coarse global prompt does not provide sufficient guidance, causing inaccurate SR results. (d) SR with Tiled Prompt (Ours): Our method of using dense, localized textual guidance proves effectively reconstructs details in videos.