ReLayout: Versatile and Structure-Preserving Design Layout Editing via Relation-Aware Design Reconstruction

TL;DR

ReLayout tackles the underexplored problem of design layout editing by enabling edits (add, delete, move, resize) that preserve the original layout structure. It introduces a relation graph to capture structural constraints and RADR to learn editing in a self-supervised manner using a multi-modal language model, thus avoiding the need for triplet training data. The framework unifies multiple editing actions under a single model and demonstrates superior editing quality and layout preservation against baselines on Crello-based data, with practical extensions to language-guided and composite editing. This work offers a foundation for more democratized, intelligent design workflows by integrating structure-aware editing into automated design tools.

Abstract

Automated redesign without manual adjustments marks a key step forward in the design workflow. In this work, we focus on a foundational redesign task termed design layout editing, which seeks to autonomously modify the geometric composition of a design based on user intents. To overcome the ambiguity of user needs expressed in natural language, we introduce four basic and important editing actions and standardize the format of editing operations. The underexplored task presents a unique challenge: satisfying specified editing operations while simultaneously preserving the layout structure of unedited elements. Besides, the scarcity of triplet (original design, editing operation, edited design) samples poses another formidable challenge. To this end, we present ReLayout, a novel framework for versatile and structure-preserving design layout editing that operates without triplet data. Specifically, ReLayout first introduces the relation graph, which contains the position and size relationships among unedited elements, as the constraint for layout structure preservation. Then, relation-aware design reconstruction (RADR) is proposed to bypass the data challenge. By learning to reconstruct a design from its elements, a relation graph, and a synthesized editing operation, RADR effectively emulates the editing process in a self-supervised manner. A multi-modal large language model serves as the backbone for RADR, unifying multiple editing actions within a single model and thus achieving versatile editing after fine-tuning. Qualitative, quantitative results and user studies show that ReLayout significantly outperforms the baseline models in terms of editing quality, accuracy, and layout structure preservation.

Figures (15)

• Figure 1: (a) Design layout editing receives an original design and an editing operation (e.g., move) as input. The red arrow indicates the moving element (orange image) and the target position. (b) Design layout editing is a challenging task. We visualize some typical undesirable results, including the low-quality layout, inaccurate editing, and destroyed layout structure. These issues are marked with red dotted boxes. (c) The proposed ReLayout enables effective design layout editing, as evidenced by the high layout quality, accurate editing, and well-preserved layout structure in the edited design.
• Figure 2: (a) An example of the relation graph. For simplicity, only a subset of nodes and edges are visualized. The node at the non-arrow end of each edge represents the source, while the node at the arrow end represents the target. (b) and (c) Heuristic rules for detecting size and position relationships. The solid-line box denotes the target node. L, R, T, B, and C stand for left, right, top, bottom, and center.
• Figure 3: Illustration of RADR, which achieves design layout editing via a self-supervised objective: reconstructing a design $\mathcal{D}$($\mathcal{A}$) from its elements $\mathcal{C}$, the relation graph $\mathcal{G}$, and an editing operation $\mathcal{O}$. An MLLM serves as the backbone, using the NLL loss on attributes $\mathcal{A}$.
• Figure 4: The reconstructed designs are visualized for qualitative comparison between our approach and GPT-4o. The ground truth (GT) designs are also shown. The three boxes on the left side of each case are the model inputs, i.e., rgb]0.87,0.81,0.99element content, rgb]0.99,0.88,0.83element relationships, and the rgb]0.80,0.94,0.98editing operation. Due to the large number of elements, we only visualize a subset of the content and relationships.
• Figure 5: Qualitative comparison of the move action. The moved element is marked with a box. The arrow points to the target position.