Representing and Modeling Inconsistent, Impossible, and Incoherent Shapes and Scenes with 2D Non-Conservative Vector Fields mapped on 2-Complexes

TL;DR

3D modeling often fails to capture expressive, impossible, or incoherent shapes common in 2D art. The paper introduces shape maps—images encoding a 2D vector field $(N_0,N_1)$ and a thickness map $T$—mapped on 2-complexes to produce view-dependent, two-sided geometry that can be ray cast to obtain real-time 3D-like effects. Key contributions include a formal representation for non-conservative vector fields, a shader-based real-time renderer for shadows/refractions, and a sketch/photo-to-shape-map pipeline utilizing 2-complex templates. This framework enables expressive depictions of 2D artworks and photographs as interactive, re-renderable 3D-like scenes, with potential extensions to web-based dynamic documents.

Abstract

In this paper, we present a framework to represent mock 3D objects and scenes, which are not 3D but appear 3D. In our framework, each mock-3D object is represented using 2D non-conservative vector fields and thickness information that are mapped on 2-complexes. Mock-3D scenes are simply scenes consisting of more than one mock-3D object. We demonstrated that using this representation, we can dynamically compute a 3D shape using rays emanating from any given point in 3D. These mock-3D objects are view-dependent since their computed shapes depend on the positions of ray centers. Using these dynamically computed shapes, we can compute shadows, reflections, and refractions in real time. This representation is mainly useful for 2D artistic applications to model incoherent, inconsistent, and impossible objects. Using this representation, it is possible to obtain expressive depictions with shadows and global illumination effects. The representation can also be used to convert existing 2D artworks into a Mock-3D form that can be interactively re-rendered.

Figures (27)

• Figure 1: Examples of static 2D pictorial documents that include incoherent, inconsistent, and impossible expressive depictions. For example, cubist artists create images based on their successive and subjective experiences in space and time gleizes1947robbins1985 that result in incoherent structures. Our approach makes it possible to turn any of such structures into dynamic ones with re-renderable elements.
• Figure 2: Examples of (a) conservative and (b) non-conservative fields. (c) demonstrate that non-conservative vector fields can produce realistic-looking rendering by converting non-conservative fields to view-dependent geometry. More interestingly, such non-conservative fields can rotate the objects behind them through refraction.
• Figure 3: An example of local and global shadows cast by a shape map mapped on a planar billboard. The mock-3D scene consists of two texture-mapped planar rectangles. Note that global shadow is volumetric even when light is in the same plane as the billboard.
• Figure 4: An example that demonstrates the advantage of 2-complexes in representing single objects. (a) shows a cartoon face represented by a single 2-manifold mesh with a boundary. This requires a complex mesh structure that should change with shape. For example, in (a) $F_4$ and $F_8$ are pentagonal patches; and $F_5$, $F_6$ and $F_{10}$ are triangles and the rest of the patches are quads. If we make the nose slightly smaller, we need to reconstruct the mesh. Using layers, it is possible to obtain general structures that do not require restructuring and/or remeshing. For example, in (b), regardless of the size of the nose, we can keep every patch as a quad. In this case, the problem is that we lose the information that the faces $F_0$ and $F_2$ share an edge. On the other hand, 2-complexes provide the best of both worlds and can describe that $F_0$, $F_2$, and $F_5$ share the same edge.
• Figure 5: Relighting Mona Lisa: an example of face relighting through channel extraction by removing shadows and shading. Note that (c) and (d) show an artificially added pearl earring as an homage to Vermeer's painting "The Girl with Pearl Earring".