DiffCSG: Differentiable CSG via Rasterization

TL;DR

An algorithm, DiffCSG, is presented, which builds upon CSG rasterization, which displays the result of boolean operations between primitives without explicitly computing the resulting mesh and, as such, bypasses black-box mesh processing.

Abstract

Differentiable rendering is a key ingredient for inverse rendering and machine learning, as it allows to optimize scene parameters (shape, materials, lighting) to best fit target images. Differentiable rendering requires that each scene parameter relates to pixel values through differentiable operations. While 3D mesh rendering algorithms have been implemented in a differentiable way, these algorithms do not directly extend to Constructive-Solid-Geometry (CSG), a popular parametric representation of shapes, because the underlying boolean operations are typically performed with complex black-box mesh-processing libraries. We present an algorithm, DiffCSG, to render CSG models in a differentiable manner. Our algorithm builds upon CSG rasterization, which displays the result of boolean operations between primitives without explicitly computing the resulting mesh and, as such, bypasses black-box mesh processing. We describe how to implement CSG rasterization within a differentiable rendering pipeline, taking special care to apply antialiasing along primitive intersections to obtain gradients in such critical areas. Our algorithm is simple and fast, can be easily incorporated into modern machine learning setups, and enables a range of applications for computer-aided design, including direct and image-based editing of CSG primitives. Code and data: https://yyyyyhc.github.io/DiffCSG/.

Figures (15)

• Figure 1: Ideation. Given two intersecting primitives (a), the result of subtracting one primitive from the other can be obtained by computing boolean operations on meshes explicitly (b). However, the topology of the resulting mesh (number of vertices, connectivity) needs to be updated for any parameter change (b-c), which is costly and complex to implement in a differentiable way. The Goldfeather algorithm displays the result of the subtraction while maintaining the original primitive meshes (d), but the discontinuity formed along the intersection of the two primitives is not anti-aliased (d, inset). We detect intersection edges explicitly and apply anti-aliasing on them (e), which allows back-propagation of gradients from pixels to primitive triangles, all the way to primitive parameters.
• Figure 2: Algorithm Overview. Our algorithm adapts a differentiable rasterization pipeline (light blue) to render CSG models (a) in a differentiable way. First, we replace the standard depth test by the Goldfeather algorithm (b), which selects among front and back faces of the CSG primitives the ones to be displayed according to boolean operations. Second, we detect intersection edges between CSG primitives (c) and provide these edges to the anti-aliasing module (d). Proper anti-aliasing is critical to allow back-propagation of gradients from the final image all the way to the primitive parameters.
• Figure 3: Goldfeather algorithm. The algorithm takes as input individual primitives of the CSG model (a). For an intersection, only the front-facing fragments that are occluded by an odd number of polygons are displayed (b, crosses depict occlusions along viewing rays). For a subtraction, the two primitives apply different parity tests against each other (c). Considering $\textcolor{red}{B} - \textcolor{blue}{A}$, the algorithm displays the front-facing fragments of $\textcolor{red}{B}$ that are occluded by an even number of polygons from $\textcolor{blue}{A}$, and the back-facing fragments of $\textcolor{blue}{A}$ that are occluded by an odd number of polygons from $\textcolor{red}{B}$.
• Figure 4: Pixel anti-aliasing.On the left, an intersection edge $(p,q)$ is formed by the top face of the gray cube and the vertical face of the blue cube. The two endpoints of the intersection edge are colored red. On the right, when considering the color blending of the crossing pixels (i.e., the gray pixel $A$ and the blue pixel $B$) of the intersection edge, two typical options may apply depending on which pixel is most covered by the edge. In the top case, edge $(p,q)$ intersects the segment connecting centers of $A, B$ inside pixel $B$, which leads to the color of $A$ blending into $B$, i.e., Color$^\textrm{B}_{\textrm{after}}$ = $\alpha$ * Color$^\textrm{A}$ + (1-$\alpha$)* Color$^\textrm{B}_{\textrm{before}}$. In the bottom case, the edge covers $A$ the most, so the color of $B$ is blended into $A$, i.e., Color$^\textrm{A}_{\textrm{after}}$ = $\alpha$ * Color$^\textrm{B}$ + (1-$\alpha$) * Color$^\textrm{A}_{\textrm{before}}$. The blending weight $\alpha$ is a linear function of the location of the crossing point, from zero at the midpoint to $0.5$ at the pixel center.
• Figure 5: Primitive tessellation. For sphere (a), the only parameter is the radius $r$. For cylinder (b), we sample $16$ points along the top and bottom circles to form the triangulation. Other than the height parameter $h$, we support taper, i.e., the top radius $r_t$ and the bottom radius $r_b$ can be different. Box (c) is the most simple primitive. We employ 12 triangles in the mesh and expose the height $h$, width $w$, and depth $d$ as the primitive parameters. For sketch-extrude (d), we resort to piecewise Bezier curves to represent the sketch (d, left) and sample $15$ points on each Bezier curve to form the triangulation after extrusion (d, right). In this case, all the control point positions (half of them are drawn) and the extrude distance $h$ are parameters.