MambaPainter: Neural Stroke-Based Rendering in a Single Step

TL;DR

This study proposes MambaPainter, capable of predicting a sequence of over 100 brush strokes in a single inference step, resulting in rapid translation and introducing a simple extension to patch-based rendering, which is used to translate high-resolution images, improving the visual quality with a minimal increase in computational cost.

Abstract

Stroke-based rendering aims to reconstruct an input image into an oil painting style by predicting brush stroke sequences. Conventional methods perform this prediction stroke-by-stroke or require multiple inference steps due to the limitations of a predictable number of strokes. This procedure leads to inefficient translation speed, limiting their practicality. In this study, we propose MambaPainter, capable of predicting a sequence of over 100 brush strokes in a single inference step, resulting in rapid translation. We achieve this sequence prediction by incorporating the selective state-space model. Additionally, we introduce a simple extension to patch-based rendering, which we use to translate high-resolution images, improving the visual quality with a minimal increase in computational cost. Experimental results demonstrate that MambaPainter can efficiently translate inputs to oil painting-style images compared to state-of-the-art methods. The codes are available at https://github.com/STomoya/MambaPainter.

Figures (3)

• Figure 1: Overview of MambaPainter. Notably, the stroke renderer can either be a neural or non-differentiable renderer at inference time.
• Figure 2: Comparison of translation results using non-overlapping patches (top) and overlapping patches (bottom).
• Figure 3: Additional translation results with different number of patches.