Single-image Reflectance and Transmittance Estimation from Any Flatbed Scanner

TL;DR

This work enables high-resolution, realistic material digitization from any flatbed scanner by decoupling shading and specular highlights through a cycle-consistent, residual delighting network that yields albedo $I_d$, followed by SVBSDF estimation to recover per-pixel opacity $O$ and transmittance $T$ along with normals and other BRDF components. The material model extends previous approaches with a thin-layer transmission component, using a physically-based BSDF $f_{l,v}^{BSDF}(A,N,S,R,O,T) = O \cdot ( \frac{A}{\pi} + s_{l,v}(N,S,R) ) + (T \cdot A)$ to capture both reflectance and transmission effects. The method is trained with cycle-consistency losses, attention-guided networks, and extensive data augmentation, and evaluated on a large scanner-based dataset with render-aware metrics (BRDF/BTDF/BSDF) and ablation studies showing the value of the delighting and residual components. The approach proves robust across devices, including smartphones, and provides a scalable path toward realistic, high-resolution material appearance suitable for SVBSDF-based rendering in virtual environments. This work promises practical impact for industries requiring accurate digital material replicas without expensive hardware or expert supervision.

Abstract

Flatbed scanners have emerged as promising devices for high-resolution, single-image material capture. However, existing approaches assume very specific conditions, such as uniform diffuse illumination, which are only available in certain high-end devices, hindering their scalability and cost. In contrast, in this work, we introduce a method inspired by intrinsic image decomposition, which accurately removes both shading and specularity, effectively allowing captures with any flatbed scanner. Further, we extend previous work on single-image material reflectance capture with the estimation of opacity and transmittance, critical components of full material appearance (SVBSDF), improving the results for any material captured with a flatbed scanner, at a very high resolution and accuracy

Figures (9)

• Figure 1: From a single image captured with any flatbed scanner (a), our method estimates a set of high-resolution SVBSDF maps (b), which can be used in any render engine (c).
• Figure 2: Some materials in our test dataset, captured on the same flatbed scanner using directional and diffuse illuminations, better suited for material capture.
• Figure 3: From an image $I_{l}$ captured with any flatbed scanner, we first estimate its albedo $I_d$ using a residual generative model $\mathcal{M}_{D}$, which removes specular highlights and shading. Taking $I_d$ as input, a second model $\mathcal{M}_{BSDF}$ estimates the rest of the SVBSDF, namely the surface normals, roughness, specular, transmittance, and opacity maps. These can be then rendered to generate photo-realistic images.
• Figure 4: Diagram of our cycle-consistent generative model capable of both material delighting and relighting.
• Figure 5: Qualitative results of our material delighting framework. On the first two rows, we show images captured with flatbed scanners under diffuse (top) and directional (bottom) illumination. We use those as input to our delighting $\mathcal{M}_D$ and relighting $\mathcal{M}_R$ models, respectively, for which we show the results on the bottom rows.