Refining Strokes by Learning Offset Attributes between Strokes for Flexible Sketch Edit at Stroke-Level

TL;DR

This work tackles stroke-level sketch edit by addressing the mismatch between source strokes and target sketches that arises from size and orientation differences. It introduces SketchMod, a refinement-driven framework that learns offset attributes—position, orientation, and scale—between strokes and uses a message-passing network to align a source stroke with target patterns before editing. The approach comprises a two-stage training regime and yields a sketch generator capable of producing both sequence- and image-formed edited sketches, outperforming baselines in edit accuracy, reconstruction quality, and human judgments. By exposing and controllably manipulating stroke attributes, SketchMod enables precise, semantically coherent, and editable stroke-level sketch edits with practical applicability in flexible sketch editing tasks.

Abstract

Sketch edit at stroke-level aims to transplant source strokes onto a target sketch via stroke expansion or replacement, while preserving semantic consistency and visual fidelity with the target sketch. Recent studies addressed it by relocating source strokes at appropriate canvas positions. However, as source strokes could exhibit significant variations in both size and orientation, we may fail to produce plausible sketch editing results by merely repositioning them without further adjustments. For example, anchoring an oversized source stroke onto the target without proper scaling would fail to produce a semantically coherent outcome. In this paper, we propose SketchMod to refine the source stroke through transformation so as to align it with the target sketch's patterns, further realize flexible sketch edit at stroke-level. As the source stroke refinement is governed by the patterns of the target sketch, we learn three key offset attributes (scale, orientation and position) from the source stroke to another, and align it with the target by: 1) resizing to match spatial proportions by scale, 2) rotating to align with local geometry by orientation, and 3) displacing to meet with semantic layout by position. Besides, a stroke's profiles can be precisely controlled during sketch edit via the exposed captured stroke attributes. Experimental results indicate that SketchMod achieves precise and flexible performances on stroke-level sketch edit.

Figures (9)

• Figure 1: (a) Two applications of stroke-level stroke edit. Stroke expansion: Adding the source stroke in red onto the target sketch. Stroke replacement: Replacing the red stroke in target by the source stroke. (b) Two approaches to refine source stroke for stroke expansion. 1) Predicting a new position to locate, which is utilized by SketchEdit li2024sketchedit and Sketch-HARP zang2025generating. 2) Refining source stroke at scale, orientation and position, allowing it to align with the target sketch's patterns for flexible sketch edit.
• Figure 2: Applying stroke-level sketch edit by SketchMod. (a) The network structure. After capturing embeddings of source stroke and the ones from the target sketch, we learn offset attributes between them to obtain a refined source stroke embedding, whose attributes are predicted and fed into a sketch generator along with all normalized strokes to generate the edited sketch. (b) Architecture of the source stroke refiner. Stroke attributes are predicted from stroke, and are utilized to learn relative offset attribute embeddings between source stroke and another. A message passing network is employed to aggregate messages from all normalized strokes by their relative offsets to finally obtain the refined source stroke embedding.
• Figure 3: The definitions of three stroke attributes: position $(a, b)$, orientation $\theta$ and scale $\bm\tau=[\tau_1, \tau_2]$.
• Figure 4: Qualitative comparisons on stroke-level sketch edit. A source stroke highlighted in red is required to be transplanted onto the target sketch via stroke replacement or expansion.
• Figure 5: Manipulating sketch synthesis at stroke-level by SketchMod.