PatternPortrait: Draw Me Like One of Your Scribbles

TL;DR

PatternPortrait addresses the challenge of generating joyful abstract portraits from photographs by combining a canny edge-based vectorization pipeline with a learning-based, pattern-driven shading model. The core method uses a graph neural network to encode and sample variations of strokes along vectorized edges via a variational graph autoencoder with a latent $z$; shading strokes are then placed in dark regions after color quantization to a small palette and rendered by a pen plotter. Key contributions include the image-to-lines pipeline, a scalable graph-based stroke representation, and a user study with approximately 280 participants validating the aesthetic and practical viability, while highlighting limitations in background noise, processing time, and shading control. The work offers a path toward co-creative, vector-based sketch generation and motivates future extension with line-arrangement learning, improved gray-value control, segmentation, and diffusion-model-based vector generation.

Abstract

This paper introduces a process for generating abstract portrait drawings from pictures. Their unique style is created by utilizing single freehand pattern sketches as references to generate unique patterns for shading. The method involves extracting facial and body features from images and transforming them into vector lines. A key aspect of the research is the development of a graph neural network architecture designed to learn sketch stroke representations in vector form, enabling the generation of diverse stroke variations. The combination of these two approaches creates joyful abstract drawings that are realized via a pen plotter. The presented process garnered positive feedback from an audience of approximately 280 participants.

Figures (8)

• Figure 1: "PatternPortrait" setup: pictures are taken with a regular webcam in SD resolution. The drawings are created with an Axidraw V3 pen plotter.
• Figure 2: Vectorization process: on the original picture a canny edge detection is applied that is then vectorized by the algorithm described in section \ref{['pictoline']}. Each vectorized line is assigned a random color for better differentiation.
• Figure 3: Graph representation of a single stroke, with the original stroke on the left. Then partitioned into $n$ points of equal distance on the stroke and finally converted to a graph structure as described in section \ref{['graphstruct']}.
• Figure 4: variational Graph autoencoder Overview
• Figure 5: Latent space observations: a model is trained with all strokes in the sample sketch shown at the top. A first (left) set is generated by observing the decoded original strokes. The second set (middle) is produced by adding noise to the original strokes latent vectors. And finally a last set (right) is created by sampling random latent vectors.