Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Authorship Attribution with Synthetic Paintings

Clarissa Loures, Caio Hosken, Luan Oliveira, Gianlucca Zuin, Adriano Veloso

TL;DR

Experimental results show that adding synthetic images leads to higher ROC-AUC and accuracy compared to using only real paintings, and this work contributes to the development of computer vision techniques for artwork authentication in data-scarce scenarios.

Abstract

Attributing authorship to paintings is a historically complex task, and one of its main challenges is the limited availability of real artworks for training computational models. This study investigates whether synthetic images, generated through DreamBooth fine-tuning of Stable Diffusion, can improve the performance of classification models in this context. We propose a hybrid approach that combines real and synthetic data to enhance model accuracy and generalization across similar artistic styles. Experimental results show that adding synthetic images leads to higher ROC-AUC and accuracy compared to using only real paintings. By integrating generative and discriminative methods, this work contributes to the development of computer vision techniques for artwork authentication in data-scarce scenarios.

Enhancing Authorship Attribution with Synthetic Paintings

TL;DR

Experimental results show that adding synthetic images leads to higher ROC-AUC and accuracy compared to using only real paintings, and this work contributes to the development of computer vision techniques for artwork authentication in data-scarce scenarios.

Abstract

Attributing authorship to paintings is a historically complex task, and one of its main challenges is the limited availability of real artworks for training computational models. This study investigates whether synthetic images, generated through DreamBooth fine-tuning of Stable Diffusion, can improve the performance of classification models in this context. We propose a hybrid approach that combines real and synthetic data to enhance model accuracy and generalization across similar artistic styles. Experimental results show that adding synthetic images leads to higher ROC-AUC and accuracy compared to using only real paintings. By integrating generative and discriminative methods, this work contributes to the development of computer vision techniques for artwork authentication in data-scarce scenarios.
Paper Structure (16 sections, 5 figures, 1 table)

This paper contains 16 sections, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Approach
  4. Dataset
  5. Sample Preparation
  6. Fine-Tuning with DreamBooth
  7. Synthetic Dataset Generation
  8. Discriminative Modeling
  9. Experiments
  10. Experimental Setup
  11. Fine-Tuning Generative Models and Image Generation
  12. Training Discriminative Models
  13. Results and Discussion
  14. Impact of the Samples
  15. Effect of Real Data Availability
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Overview of the image generation pipeline. Starting from Gaussian noise and a prompt (e.g., "a full owhx painting"), the CLIP encoder conditions the U-Net denoising process in latent space. After multiple refinement steps, the latent image is decoded by a VAE into a full-resolution painting. This process is repeated to generate a synthetic dataset per artist.
  • Figure 2: Patch sampling strategies. (A) Original image. (B) M1: baseline sampling with moderate overlap. (C) M2: denser sampling obtained by doubling the number of patches along both axes. Axis color reflects the level of overlap between samples.
  • Figure 3: Examples of three synthetic images generated for each artist. The images reflect stylistic variations captured by the model, showcasing differences in composition, subject matter, and brushwork. Despite efforts to mitigate generation biases, some partial figures and cropped elements remain, likely influenced by the characteristics of the training dataset.
  • Figure 4: Heatmap of ROC-AUC values for different training settings. The figure illustrates the variation in the model's ability to discriminate between classes depending on the dataset used for training and testing. Higher values indicate better discrimination between authors.
  • Figure 5: Heatmap of Accuracy values across training settings. The figure highlights differences in overall classification performance, showing how accuracy varies depending on whether the model was trained with real, synthetic, or mixed data.