Trajectory-Guided Diffusion for Foreground-Preserving Background Generation in Multi-Layer Documents

TL;DR

This work reframes diffusion-based document background generation as trajectory design in a structured latent space to preserve foreground content while maintaining cross-page stylistic consistency. It introduces a Style Bank of persistent latent directions and a Diffusion State-Space Control (SSC) mechanism that attenuates updates in foreground regions via a time-gated, energy-based approach, all without explicit masking or retraining. By coupling global style directions with a thermodynamic interpretation of latent stabilization, the method achieves foreground preservation and multi-page coherence across diverse document layouts, demonstrated through curated datasets, qualitative examples, and robust user studies. The approach is backbone-agnostic and training-free, offering a principled framework for reliable, structured generative modeling in multi-layer documents with practical impact for document editing and visual design workflows.

Abstract

We present a diffusion-based framework for document-centric background generation that achieves foreground preservation and multi-page stylistic consistency through latent-space design rather than explicit constraints. Instead of suppressing diffusion updates or applying masking heuristics, our approach reinterprets diffusion as the evolution of stochastic trajectories through a structured latent space. By shaping the initial noise and its geometric alignment, background generation naturally avoids designated foreground regions, allowing readable content to remain intact without auxiliary mechanisms. To address the long-standing issue of stylistic drift across pages, we decouple style control from text conditioning and introduce cached style directions as persistent vectors in latent space. Once selected, these directions constrain diffusion trajectories to a shared stylistic subspace, ensuring consistent appearance across pages and editing iterations. This formulation eliminates the need for repeated prompt-based style specification and provides a more stable foundation for multi-page generation. Our framework admits a geometric and physical interpretation, where diffusion paths evolve on a latent manifold shaped by preferred directions, and foreground regions are rarely traversed as a consequence of trajectory initialization rather than explicit exclusion. The proposed method is training-free, compatible with existing diffusion backbones, and produces visually coherent, foreground-preserving results across complex documents. By reframing diffusion as trajectory design in latent space, we offer a principled approach to consistent and structured generative modeling.

Figures (36)

• Figure 1: Overview of our document-centric foreground-aware background generation Given structured multi-page documents (e.g., PDF pages), we extract page-level content and layout cues, including Document Summarized Details and Foreground Region Extraction using text-region bounding boxes. Together with a user-provided Background Style (or customized style prompt) that selects a fixed latent direction from an internal Style Bank$\mathcal{S}=\{s_i\}$. We then synthesize visually consistent backgrounds using a pretrained text-to-image diffusion model instantiated with Diffusion State-Space Control (SSC) as a concrete control mechanism. SSC modifies the diffusion trajectory in latent space by applying (i) Global Style Control, which injects the selected style direction into the latent state as $x_t \leftarrow x_t + \lambda_s s$, and (ii) Foreground Preservation, using a time-dependent schedule to progressively constrain foreground tokens associated with text regions, while keeping background tokens expressive throughout denoising. The generated background layers are finally composited with the original document foreground to produce coherent, readable, and style-consistent multi-page documents.
• Figure 2: Representative qualitative comparison onOurs, BAGEL, and GPT-5. Rows correspond to style conditions (Textures). BAGEL fails to preserve the foreground text. While GPT-5 visually appealing designs, violates the editing setting by (i) altering the original layout, (ii) modifying or replacing existing figures, and (iii) hallucinating additional text not present in the input document, making it unsuitable for foreground-preserving document editing. More results in the supplementary materials.
• Figure 3: Ablation study on document-centric foreground-aware background generation.Ours (top) preserves text readability and layout fidelity while maintaining a consistent visual style across pages. Ours w/o Style Bank (middle) removes the persistent style direction, leading to noticeable variation in background appearance across pages despite preserving foreground readability. Ours w/o SSC disables diffusion state-space control for foreground stabilization, causing background textures to intrude into text regions and significantly degrading readability and accessibility. These results highlight the complementary roles of the Style Bank in enforcing cross-page stylistic consistency and SSC in ensuring foreground preservation during diffusion.
• Figure 4: Results of the user study. Thirty participants compared three anonymized background generation methods across four evaluation criteria: Layout preservation, Color harmony, Graphic style consistency, and Prompt compliance (left). Our approach consistently received the highest average ratings across all dimensions. In the overall preference comparison (right), 86.9% of votes favored our method, compared to 13.1% for GPT-5.
• Figure 5: Visual comparison on academic-format PDFs (A4). Each row corresponds to a different style setting (Colorful, Geometric, Muted, Professional, Real & Natural, Shapes, Textures).