Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nash Meets Wertheimer: Using Good Continuation in Jigsaw Puzzles

Marina Khoroshiltseva, Luca Palmieri, Sinem Aslan, Sebastiano Vascon, Marcello Pelillo

TL;DR

A new challenging version of the puzzle solving problem in which one deliberately ignores conventional color and shape features and relies solely on the presence of linear geometrical patterns, which shows the intrinsic complexity of the purely line-based puzzle problem as well as the relative effectiveness of the game-theoretic formulation.

Abstract

Jigsaw puzzle solving is a challenging task for computer vision since it requires high-level spatial and semantic reasoning. To solve the problem, existing approaches invariably use color and/or shape information but in many real-world scenarios, such as in archaeological fresco reconstruction, this kind of clues is often unreliable due to severe physical and pictorial deterioration of the individual fragments. This makes state-of-the-art approaches entirely unusable in practice. On the other hand, in such cases, simple geometrical patterns such as lines or curves offer a powerful yet unexplored clue. In an attempt to fill in this gap, in this paper we introduce a new challenging version of the puzzle solving problem in which one deliberately ignores conventional color and shape features and relies solely on the presence of linear geometrical patterns. The reconstruction process is then only driven by one of the most fundamental principles of Gestalt perceptual organization, namely Wertheimer's {\em law of good continuation}. In order to tackle this problem, we formulate the puzzle solving problem as the problem of finding a Nash equilibrium of a (noncooperative) multiplayer game and use classical multi-population replicator dynamics to solve it. The proposed approach is general and allows us to deal with pieces of arbitrary shape, size and orientation. We evaluate our approach on both synthetic and real-world data and compare it with state-of-the-art algorithms. The results show the intrinsic complexity of our purely line-based puzzle problem as well as the relative effectiveness of our game-theoretic formulation.

Nash Meets Wertheimer: Using Good Continuation in Jigsaw Puzzles

TL;DR

A new challenging version of the puzzle solving problem in which one deliberately ignores conventional color and shape features and relies solely on the presence of linear geometrical patterns, which shows the intrinsic complexity of the purely line-based puzzle problem as well as the relative effectiveness of the game-theoretic formulation.

Abstract

Jigsaw puzzle solving is a challenging task for computer vision since it requires high-level spatial and semantic reasoning. To solve the problem, existing approaches invariably use color and/or shape information but in many real-world scenarios, such as in archaeological fresco reconstruction, this kind of clues is often unreliable due to severe physical and pictorial deterioration of the individual fragments. This makes state-of-the-art approaches entirely unusable in practice. On the other hand, in such cases, simple geometrical patterns such as lines or curves offer a powerful yet unexplored clue. In an attempt to fill in this gap, in this paper we introduce a new challenging version of the puzzle solving problem in which one deliberately ignores conventional color and shape features and relies solely on the presence of linear geometrical patterns. The reconstruction process is then only driven by one of the most fundamental principles of Gestalt perceptual organization, namely Wertheimer's {\em law of good continuation}. In order to tackle this problem, we formulate the puzzle solving problem as the problem of finding a Nash equilibrium of a (noncooperative) multiplayer game and use classical multi-population replicator dynamics to solve it. The proposed approach is general and allows us to deal with pieces of arbitrary shape, size and orientation. We evaluate our approach on both synthetic and real-world data and compare it with state-of-the-art algorithms. The results show the intrinsic complexity of our purely line-based puzzle problem as well as the relative effectiveness of our game-theoretic formulation.

Paper Structure

This paper contains 20 sections, 1 theorem, 13 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Jigsaw Puzzles and Nash Equilibria
  4. Nash equilibria
  5. Jigsaw puzzle games
  6. Finding Nash equilibria
  7. Payoff functions
  8. Partial payoff matrix
  9. Line-based Compatibility Measure
  10. Experiments
  11. Datasets
  12. Synthetic Lines Generation
  13. Puzzles from Real Data
  14. Pieces Creation
  15. Evaluation metrics
  16. ...and 5 more sections

Key Result

theorem thmcountertheorem

A point $x\in\Theta$ is a limit trajectory of (eq:repdyncont) starting from the interior of $\Theta$ if and only if $x$ is a Nash equilibrium. Further, if point $x\in\Theta$ is a strict Nash equilibrium then it is also asymptotically stable.

Figures (6)

  • Figure 1: Three groups of fragments from an ancient broken fresco. Here, the good continuation principle plays a major role in the reassembly process.
  • Figure 2: The difficulty of solving purely line-based jigsaw puzzles. The reader is invited to solve this small ($3 \times 3$) instance of the problem. Our algorithm solves it perfectly.
  • Figure 3: The three regions of the space : neutral (yellow), adjacent (green) and overlapping (red).
  • Figure 4: An example of the computation of the cost for the players $i$ and $j$ given a candidate $\gamma$ strategy. The two sets $L_{i|\gamma}$ and $L_{j|\gamma}$ are created by selecting only lines which intersects the opposite player, and the total cost consists of the sum of the cost of the corresponding LAP and the penalty for unmatched lines.
  • Figure 5: Some examples of puzzles using real vector maps from Openstreetmaps1.
  • ...and 1 more figures

Theorems & Definitions (1)

  • theorem thmcountertheorem