Compositing with 2D Vector Fields by using Shape Maps that can represent Inconsistent, Impossible, and Incoherent Shapes

TL;DR

Shape maps encode a 2D shape and thickness via a 2D vector field $(x(u,v), y(u,v))$ and thickness $d(u,v)$, enabling image-based reflection, refraction, and Fresnel effects without 3D rendering. The core contribution is a reformulated compositing equation $CI = \alpha FI + (1-\alpha) \big( f\,EI(R) + (1-f)\,BI(T) \big)$, where $R$ and $T$ are shape-map-driven mappings and $f$ is a Fresnel term, reducing the process to a single additional layer. Artists can paint shape maps directly in 2D (as $(r,g,b) = (\tfrac{1}{2}(x+1), \tfrac{1}{2}(y+1), d)$) and adjust parameters such as the index proxy $a$ and thickness $d$ to produce realistic or painterly effects with optional environment maps. The method supports impossible/incoherent shapes, enables glossiness via mipmapped environment maps, and can be integrated into standard 2D workflows for fast, dynamic editing of reflections, refractions, translucency, and Fresnel control. This yields a practical, painter-friendly alternative to 3D rendering for stylized 2D composites that still convey convincing optical cues.

Abstract

In this paper, we present a new compositing approach to obtain stylized reflections and refractions with a simple control. Our approach does not require any mask or separate 3D rendering. Moreover, only one additional image is sufficient to obtain a composited image with convincing qualitative reflection and refraction effects. We have also developed linearized methods that are easy to compute. Although these methods do not directly correspond to the underlying physical phenomena of reflection and refraction, they can provide results that are visually similar to realistic 3D rendering. The main advantage of this approach is the ability to treat images as ``mock-3D'' shapes that can be inserted into any digital paint system without any significant structural change. The core of our approach is the shape map, which encodes 2D shape and thickness information for all visible points of an image of a shape. This information does not have to be complete or consistent to obtain interesting composites. In particular, the shape maps allow us to represent impossible and incoherent shapes with 2D non-conservative vector fields.

Figures (12)

• Figure 1: An example of compositing with 2D vector fields by using shape maps. The particular shape map in (b) is inspired by a photograph of a real bottle in (a). This example demonstrates that a single image included in a 2D digital painting program can be sufficient to obtain a wide variety of refraction and reflection effects. (d) Reflection and refraction combined with the Fresnel term. (f) Glossy reflection and translucent refraction combined with a Fresnel. (f) directly shows Fresnel effects using a black background and a white environment map.
• Figure 2: Some of the layers that are needed to obtain a final compositing image.
• Figure 3: An example of the effect of global $alpha$ with non-photo-realistic compositing with reflection, glossy reflection, refraction, and translucence combined with Fresnel. (a) Hand-drawn shape map and (b) foreground image. The white regions of the foreground image are transparent, i.e. $\alpha_{FI}=0$. In other regions, $\alpha_{FI}=1$. The images in (c), (d), and (e) show the effect of global $\alpha_G$.
• Figure 4: Examples of painterly filter effect obtained by our compositing equation using with variety of shape maps. In these examples, we use only a background image and a shape map. One can create more complicated images by including a reflection from the environment map and a foreground image. Using this approach, it is also possible to obtain akleman2023recursive.
• Figure 5: An example of reflection and refraction with an impossible object. The shape map and the foreground image were painted by an artist in a digital painting program. We use the global $\alpha_G=0.5$ to make the foreground layer slightly transparent. Note that there exists no continuous height field that can produce these vector fields.