Generative Whimsy Jigsaws

Abstract

Unlike conventional jigsaw puzzles which partition an image into a predefined set of rectangular pieces, ``whimsy’’ jigsaw puzzles depict a variety of objects, such as animals, vehicles, and characters, through the outlines of the pieces themselves. Although skilled artists have created visually compelling and playful puzzles within this framework, designing whimsy jigsaws remains highly challenging—creators must simultaneously partition a given outline into multiple interlocking pieces while ensuring that each piece conveys a recognizable visual concept. This difficulty limits accessibility, preventing casual creation by less experienced users, particularly when designing puzzles with many pieces and concepts. To address this challenge, we present a computational framework for automatically designing whimsy jigsaws from a loosely specified set of visual targets, each assigned to an individual piece. Specifically, we take a set of text prompts as input and use differentiable optimization to refine the geometry of each piece, guided by a pre-trained diffusion model. Since naive formulations suffer from a fundamental tension in whimsy jigsaws between geometric validity and recognizability, this paper develops a representation for planar partitioning that guarantees the partitioning of the domain into a specified number of connected pieces, while allowing for highly flexible geometric evolution. The key component is the scalar slowness field, which is used to compute a warped geodesic distance field given a point source by solving the Eikonal equation. Given distance fields from multiple sources, we can compute warped Voronoi cells that define pieces of a puzzle. The representation is fully differentiable, with gradients computed via adjoint sensitivity analysis, and the slowness field is represented as a neural implicit field. Experimental results demonstrate that our framework generates diverse, thematically consistent jigsaw puzzles across a wide range of user prompts, enabling the automated design of puzzles with complex, figurative pieces.

Kenji Tojo

Ph.D Course

Nobuyuki Umetani

Associate Professor

My research interests include interactive smart engineering design tool using physics simulation and machine learning.