Stroke2Sketch: Harnessing Stroke Attributes for Training-Free Sketch Generation

TL;DR

Stroke2Sketch introduces a training-free framework that transfers reference stroke attributes to a content image while preserving structure, achieved through cross-image stroke attention within pretrained diffusion models. It deploys three novel components—Directive Attention Module for foreground-focused stylization, Semantic Preservation Module for contour- and text-guided semantic alignment, and Stroke Details Propagation Enhancement for refined stroke details—along with adaptive contrast and CFG-based denoising. The method demonstrates state-of-the-art performance on Stroke2Sketch-dataset and FS2K, with strong user preferences and robust ablations validating the contributions. By leveraging pretrained priors and principled feature interactions, Stroke2Sketch enables high-fidelity, reference-aligned sketches without style-specific training, offering practical utility for artistic and design workflows.

Abstract

Generating sketches guided by reference styles requires precise transfer of stroke attributes, such as line thickness, deformation, and texture sparsity, while preserving semantic structure and content fidelity. To this end, we propose Stroke2Sketch, a novel training-free framework that introduces cross-image stroke attention, a mechanism embedded within self-attention layers to establish fine-grained semantic correspondences and enable accurate stroke attribute transfer. This allows our method to adaptively integrate reference stroke characteristics into content images while maintaining structural integrity. Additionally, we develop adaptive contrast enhancement and semantic-focused attention to reinforce content preservation and foreground emphasis. Stroke2Sketch effectively synthesizes stylistically faithful sketches that closely resemble handcrafted results, outperforming existing methods in expressive stroke control and semantic coherence. Codes are available at https://github.com/rane7/Stroke2Sketch.

Figures (20)

• Figure 1: (a) and (b) show results from training-based methods (Ref2Sketch and Semi-Ref2Sketch), which struggle with unseen styles, leading to poor content alignment. (c) and (d) illustrate IP-Adapter and InstantStyle results, retaining style but lacking content alignment. (e) to (g) show ControlNet-based methods, preserving content but failing to match reference styles. (h) shows our method’s output, achieving superior content preservation and style alignment without using ControlNet. Detailed prompts are shown in the supplementary materials.
• Figure 2: The Stroke2Sketch architecture. The content image $I^{\text{cnt}}$ undergoes contour detection, generating feature representations $z^{\text{cont}}$ and $z^{\text{cnt}}$, while the reference sketch $I^{\text{ref}}$ is inverted to produce $z^{\text{ref}}$. The Directive Attention Module (DAM) aligns high-level semantic features between the content and reference features by emphasizing cross-image semantic correspondences $A_{n}$. Self-attention maps $F_{SA}$ are aggregated and clustered to produce segmentation masks $M_{i}$, which help distinguish foreground from background regions. Cross-image Stroke Attention (CSA) transfers stroke attributes, and the Semantic Preservation Module (SPM) enforces semantic alignment and stroke fidelity in the generated sketch $\hat{I}^{\text{ske}}$ via loss $\mathcal{L}_{\text{sem}}$ and contour-based structural integration. Lastly, Stroke Detail Propagation Enhancement (SDPE) refines details, resulting in the final output sketch $I^{\text{ske}}$. SD represents the pre-trained diffusion model.
• Figure 3: Stroke alignment results with CSA. The feature images are shown in the first row. The second row shows results where feature alignment between the key and value exchange is applied.
• Figure 4: Examples of DAM in action. The first column shows the content images, the second column displays segmentation maps obtained by clustering self-attention maps, the third column provides the reference sketches, and the fourth column illustrates the sketches generated by DAM after applying stroke attribute transfer and segmentation.
• Figure 5: Comparison of sketches generated using DAM. The top row shows Cnt. and Ref. images with stroke styles reflecting specific high-level attributes such as hair, eyebrows, and abstracted clothing texture. The second to fourth columns display sketches generated to match the content images in the top row, each adopting the stroke styles and high-level features from the reference. The bottom row highlights zoomed-in areas to emphasize the transferred stroke attributes.