Solving Convex Partition Visual Jigsaw Puzzles

TL;DR

This paper introduces Convex Partition puzzles, a broad generalization of polygonal jigsaw problems, and develops a greedy solver that fuses geometric and pictorial cues to reconstruct shuffled convex faces from a seed point-set. The approach builds a mating graph, enriches it with pictorial compatibility via extrapolated content, and then uses a two-phase spring-mass optimization to recover piece poses, followed by a cycle-based global assembly to form aggregates and a final consistent layout. A first benchmark dataset of 75 convex partition puzzles (across noiseless and noisy variants) is released, and the solver is evaluated against ground-truth matings and poses, showing robust performance with graceful degradation under noise and compatibility with CC puzzles. The work broadens puzzle-solving applicability to real-world domains like archaeology and document restoration, and paves the way for larger-scale and data-driven extensions.

Abstract

Jigsaw puzzle solving requires the rearrangement of unordered pieces into their original pose in order to reconstruct a coherent whole, often an image, and is known to be an intractable problem. While the possible impact of automatic puzzle solvers can be disruptive in various application domains, most of the literature has focused on developing solvers for square jigsaw puzzles, severely limiting their practical use. In this work, we significantly expand the types of puzzles handled computationally, focusing on what is known as Convex Partitions, a major subset of polygonal puzzles whose pieces are convex. We utilize both geometrical and pictorial compatibilities, introduce a greedy solver, and report several performance measures next to the first benchmark dataset of such puzzles.

Figures (8)

• Figure 1: Convex partitions and puzzles. (A) A Convex Partition graph superimposed on an image of a Pompeii fresco (free photo from Wikimedia, by Marie-Lan Nguyen). In such depictions, it is easy to see how each edge of each face/piece matches at most one edge of a different face/piece. (B) A Convex Partition puzzle created from the convex partition by randomly shuffling and transforming the pieces. The challenge of Convex Partition puzzle solvers is to reconstruct the coherent structure and image from A.
• Figure 2: The mating graph of the convex partition from Fig. \ref{['fig:convex_partition_example']}. The nodes (i.e., the edges of the pieces) are depicted as black points. Piece and mating links are colored red and blue, respectively. (A): A noiseless nonaccidental puzzle, where edges of pieces have at most one mating link. (B): A noisy puzzle, were nodes might have many mating links. The full mating graph for this noisy puzzle contains many more mating links than illustrated, but to avoid clutter we show only a subset.
• Figure 3: Pictorial extrapolation and compatibility. (A) Extrapolation of pictorial data into the eroded zone (and beyond) using Stable Diffusion Rombach_2022_CVPR. The extrapolated band is shown in lighted shades for clarity). Fragment taken from a recent fresco fragment dataset elkin2025recognizing. (B) The compatibility of a mating link, in this case between the two vertical edges, is computed between $X(e_k^j,w)$ and $O(e_y^u,w)$, depicted in green and red, respectively. Note that the former is extrapolated data while the latter is genuine. For clarify the sketch uses $w=2$ though in practice it was larger. The colored (orange) region demarcates the genuine data of the eroded piece while the extrapolated band is in grayscale. Here, $l(e_y^u)$ and $l(e_j^k)$ are roughly equal.
• Figure 4: Illustration of matching vertices of the mating edges $(v_k^j,v_{k+1}^j)$ and $(v_l^u,v_{l+1}^u)$ of the pieces $p_l$ (blue) and $p_k$ (gray), respectively. The corresponding distances/springs between vertices) are colored red. (A): The (correct) mating vertices $v_k^j$, $v_l^u$ and $v_{k+1}^j$,$v_{l+1}^u$. (B): Incorrectly matching $v_{k+1}^j$ to $v_l^u$ and $v_k^j$ to $v_{l+1}^u$ entails overlap between the pieces when the distance between vertices (or equally, the energy of the connecting springs) is minimized. (C): Incorrect matching is possible without overlaps only if the combined distances (i.e., spring lengths) is non negligible.
• Figure 5: Illustration for the springs-mass system simulation on four pieces . (A): The initial state of the pieces and a pair of springs for each mating link that is considered. (B): An intermediate state of the pieces as they get closer to their final pose while artificially allowed to overlap each other. (C): The final state of the bodies after convergence and while overlapping is no longer allowed.