Fiber-level Woven Fabric Capture from a Single Photo

TL;DR

The paper tackles the challenge of rendering fabrics with realistic micro-structure by reconstructing fiber-level geometry and appearance from a single microscope image. It introduces a unified pipeline: neural-network-based parameter initialization, joint geometry-appearance optimization via differentiable rasterization, and subsequent appearance refinement via differentiable path tracing. A key contribution is a fiber-level procedural geometry model paired with an approximated shading model and a high-performance differentiable renderer, enabling efficient optimization and high-fidelity rendering at both distant and close-up views. The framework also implements a patch-space, two-scale path-tracing approach to render large fabric scenes efficiently. Overall, the method enables plausible fiber-level fabric capture and rendering from minimal input, with potential for downstream editing and broader fabric types in future work.

Abstract

Accurately rendering the appearance of fabrics is challenging, due to their complex 3D microstructures and specialized optical properties. If we model the geometry and optics of fabrics down to the fiber level, we can achieve unprecedented rendering realism, but this raises the difficulty of authoring or capturing the fiber-level assets. Existing approaches can obtain fiber-level geometry with special devices (e.g., CT) or complex hand-designed procedural pipelines (manually tweaking a set of parameters). In this paper, we propose a unified framework to capture fiber-level geometry and appearance of woven fabrics using a single low-cost microscope image. We first use a simple neural network to predict initial parameters of our geometric and appearance models. From this starting point, we further optimize the parameters of procedural fiber geometry and an approximated shading model via differentiable rasterization to match the microscope photo more accurately. Finally, we refine the fiber appearance parameters via differentiable path tracing, converging to accurate fiber optical parameters, which are suitable for physically-based light simulations to produce high-quality rendered results. We believe that our method is the first to utilize differentiable rendering at the microscopic level, supporting physically-based scattering from explicit fiber assemblies. Our fabric parameter estimation achieves high-quality re-rendering of measured woven fabric samples in both distant and close-up views. These results can further be used for efficient rendering or converted to downstream representations. We also propose a patch-space fiber geometry procedural generation and a two-scale path tracing framework for efficient rendering of fabric scenes.

Figures (24)

• Figure 1: Our capture configuration for real fabric data. We use a microscope camera to capture the fabric samples under one of the light sources inside the microscope, blocking off other lights. The fabric sample is placed on a plane under the camera. We measure the distances between all elements, as well as the camera field of view, so we can reconstruct the same setup in synthetic renderings.
• Figure 2: (a) Woven fabric consists of weft and warp yarns. (b) The position of each yarn is coarsely defined by a centerline curve. The shape of a centerline curve is defined by several parameters: the maximum inclination angle $u_\mathrm{max}$, the length of a yarn segment, and the radius of the circle. (c) Around each centerline curve, a set of fibers is generated.
• Figure 3: Comparison of different curves. The last column shows the ground truth obtained by microscope capture Sadeghi:2013:Cloth, with purple lines indicating the centerlines of the yarns, which are treated as the ground truth. For plain, parabola-weighted circular function causes the top of the yarn to be too flat. For satin, the circular function leads to unrealistic arching at the top of the yarn. The hybrid way avoids these issues.
• Figure 4: With different parameters ($u_\mathrm{max}$, $l$, and $\beta$), our centerline formulation Eqn. (\ref{['eq:centerline']}) exhibits various shapes. The baseline settings are $u_\mathrm{max}=0.5\pi$, $l=2$ and $\beta = 1.0$.
• Figure 5: The effects of different $\kappa$ on the fiber curve. Our azimuthal noise causes the highest point of the fiber to move in the azimuthal direction. The farther $\kappa$ deviates from 1, the greater the movement.