Stroke Patches: Customizable Artistic Image Styling Using Regression

TL;DR

This work tackles artistic image rendering with explicit control over stroke composition and detail by introducing stroke patches—procedurally generated stroke primitives with tunable attributes—and a regression-based U-Net that maps continuous image tones to discrete strokes. The method yields a range of patch-driven styles and supports scaling and parameter-driven adjustments, offering a controllable alternative to style transfer and diffusion-based approaches. The main contributions are the stroke patch concept and the regression mapping framework, demonstrated on diverse inputs with open-source code for reproducibility. The approach enables artists to design bespoke stroke grammars, bridging procedural generation and learned rendering.

Abstract

We present a novel, regression-based method for artistically styling images. Unlike recent neural style transfer or diffusion-based approaches, our method allows for explicit control over the stroke composition and level of detail in the rendered image through the use of an extensible set of stroke patches. The stroke patch sets are procedurally generated by small programs that control the shape, size, orientation, density, color, and noise level of the strokes in the individual patches. Once trained on a set of stroke patches, a U-Net based regression model can render any input image in a variety of distinct, evocative and customizable styles.

Figures (6)

• Figure 1: Example stroke patches.
• Figure 2: Training (top) The training set is a set of stroke patches $P$. Each stroke patch $p_n$ has noise $\mathcal{N}$ added and then is blurred with a Gaussian kernel $b$ and input to U-Net $f_\theta$. The output of these steps is then used to compute the MSE loss with respect to $p_n$. Inference (bottom) An arbitrary input image $x$ is passed through the trained U-Net $f_\theta$ yielding a stylized image $y$.
• Figure 3: Results Part 1. The top row depicts three original unprocessed images and the middle and bottom rows show the results of applying the model trained on two different stroke patches sets (Speedball Pen and Wet Brush). A sample stroke patch is shown to the left of the stylized images. Original photo credits: Ian Jaffray (left, center), Mbdortmund, GFDL 1.2 http://www.gnu.org/licenses/old-licenses/fdl-1.2.html, via Wikimedia Commons https://commons.wikimedia.org/wiki/File:Wuppertal-100508-12825-Uferstra%C3%9Fe.jpg (right).
• Figure 4: Results Part 2. Each row shows the results of applying the model trained on three different stroke patches sets (Diamond Brush, Cuneiform Brush, and Scribble Pencil) using the same original images as \ref{['fig:results_1']}. A sample stroke patch is shown to the left of the stylized images.
• Figure 5: Effect of scaling the original images. Each column shows the effect of reducing the original image size by the percentage specified in the column heading before applying the model trained on the Wet Brush (top row) and Speedball Pen (bottom row) stroke patch sets and then enlarging the resulting image back to its original size.