Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Benchmarking Content-Based Puzzle Solvers on Corrupted Jigsaw Puzzles

Richard Dirauf, Florian Wolz, Dario Zanca, Björn Eskofier

TL;DR

The paper addresses the robustness gap of content-based jigsaw puzzle solvers when faced with realistic artefact corruptions such as missing pieces, eroded edges, and eroded contents. It benchmarks five state-of-the-art solvers—three heuristics and two deep-learning models (Transformer and Positional Diffusion)—on corrupted puzzles derived from the PuzzleWikiArts dataset, using two established metrics. The study finds that heuristic solvers remain relatively robust to some corruptions, but their performance degrades sharply with increased corruption, while deep learning models improve substantially with data augmentation, with Positional Diffusion showing the strongest overall robustness across most scenarios. These results suggest that advanced diffusion-based approaches, possibly in hybrid configurations with shape-aware methods, hold the most promise for automated reconstruction of real-world fragmented artefacts, and they highlight directions for creating more realistic datasets and hybrid modeling approaches.

Abstract

Content-based puzzle solvers have been extensively studied, demonstrating significant progress in computational techniques. However, their evaluation often lacks realistic challenges crucial for real-world applications, such as the reassembly of fragmented artefacts or shredded documents. In this work, we investigate the robustness of State-Of-The-Art content-based puzzle solvers introducing three types of jigsaw puzzle corruptions: missing pieces, eroded edges, and eroded contents. Evaluating both heuristic and deep learning-based solvers, we analyse their ability to handle these corruptions and identify key limitations. Our results show that solvers developed for standard puzzles have a rapid decline in performance if more pieces are corrupted. However, deep learning models can significantly improve their robustness through fine-tuning with augmented data. Notably, the advanced Positional Diffusion model adapts particularly well, outperforming its competitors in most experiments. Based on our findings, we highlight promising research directions for enhancing the automated reconstruction of real-world artefacts.

Benchmarking Content-Based Puzzle Solvers on Corrupted Jigsaw Puzzles

TL;DR

The paper addresses the robustness gap of content-based jigsaw puzzle solvers when faced with realistic artefact corruptions such as missing pieces, eroded edges, and eroded contents. It benchmarks five state-of-the-art solvers—three heuristics and two deep-learning models (Transformer and Positional Diffusion)—on corrupted puzzles derived from the PuzzleWikiArts dataset, using two established metrics. The study finds that heuristic solvers remain relatively robust to some corruptions, but their performance degrades sharply with increased corruption, while deep learning models improve substantially with data augmentation, with Positional Diffusion showing the strongest overall robustness across most scenarios. These results suggest that advanced diffusion-based approaches, possibly in hybrid configurations with shape-aware methods, hold the most promise for automated reconstruction of real-world fragmented artefacts, and they highlight directions for creating more realistic datasets and hybrid modeling approaches.

Abstract

Content-based puzzle solvers have been extensively studied, demonstrating significant progress in computational techniques. However, their evaluation often lacks realistic challenges crucial for real-world applications, such as the reassembly of fragmented artefacts or shredded documents. In this work, we investigate the robustness of State-Of-The-Art content-based puzzle solvers introducing three types of jigsaw puzzle corruptions: missing pieces, eroded edges, and eroded contents. Evaluating both heuristic and deep learning-based solvers, we analyse their ability to handle these corruptions and identify key limitations. Our results show that solvers developed for standard puzzles have a rapid decline in performance if more pieces are corrupted. However, deep learning models can significantly improve their robustness through fine-tuning with augmented data. Notably, the advanced Positional Diffusion model adapts particularly well, outperforming its competitors in most experiments. Based on our findings, we highlight promising research directions for enhancing the automated reconstruction of real-world artefacts.

Paper Structure

This paper contains 18 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Content-Based Heuristic Jigsaw Puzzle Solvers
  4. Content-Based Deep Learning Jigsaw Puzzle Solvers
  5. Methodology
  6. Types of Corruption for Jigsaw Puzzles
  7. Missing Pieces
  8. Eroded Edges
  9. Eroded Contents
  10. Content-Based Jigsaw Puzzle Dataset
  11. Experimental Design
  12. Results
  13. Baseline: Standard Jigsaw Puzzles
  14. Experiment 1: Corrupting Jigsaw Puzzles with Missing Pieces
  15. Experiment 2: Corrupting Jigsaw Puzzles with Eroded Edges
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Example of a $12 \times 12$ puzzle modified by each of the three corruption types. The original image is taken from the PuzzleWikiArts dataset talon_ganzzle_2022.
  • Figure 2: Results of Experiment 1 showing the performance of all solvers on puzzles with missing pieces for four different puzzle sizes.
  • Figure 3: Results of Experiment 2 showing the performance of all solvers on puzzles with eroded edges for four different puzzle sizes.
  • Figure 4: Results of Experiment 3 showing the performance of all solvers on puzzles with eroded contents for four different puzzle sizes.