Digital Kitchen Remodeling: Editing and Relighting Intricate Indoor Scenes from a Single Panorama

TL;DR

This work tackles editing and relighting intricate indoor spaces from a single panorama by leveraging a single-camera HDR capture setup to reconstruct a complete 3D scene and an environment map for physically-based relighting. It contributes a calibrated HDR pipeline with a low-cost photometric calibration method, a 141-scene Pano-Pano HDR dataset, and a kitchen component dataset to enable editable materials and lighting. An end-to-end pipeline is demonstrated for automatic kitchen layout estimation, object insertion, and Mitsuba-based rendering to produce realistic virtual stagings suitable for real estate and design exploration. The approach advances affordable, photometrically accurate virtual staging from minimal data, though it currently focuses on a limited set of kitchen layouts and materials, with future work aimed at expanding coverage and validating usability.

Abstract

We present a novel virtual staging application for kitchen remodeling from a single panorama. To ensure the realism of the virtual rendered scene, we capture real-world High Dynamic Range (HDR) panoramas and recover the absolute scene radiance for high-quality scene relighting. Our application pipeline consists of three key components: (1) HDR photography for capturing paired indoor and outdoor panoramas, (2) automatic kitchen layout generation with new kitchen components, and (3) an editable rendering pipeline that flexibly edits scene materials and relights the new virtual scene with global illumination. Additionally, we contribute a novel Pano-Pano HDR dataset with 141 paired indoor and outdoor panoramas and present a low-cost photometric calibration method for panoramic HDR photography.

Figures (10)

• Figure 1: Kitchen Remodeling Application: (Left) A captured indoor panorama showcases an intricate existing scene. (Right) The kitchen area is virtually remodeled with new walls, cabinets and appliances, and is remodeled within the existing outdoor context with additional electrical lighting.
• Figure 2: Overview of Our Approach: We present a comprehensive pipeline from data collection to scene editing and relighting. First, a single Ricoh Theta Z1 camera captures paired indoor and outdoor panoramas. Second, the indoor panorama is used to estimate the 3D room layout, and the outdoor panorama provides a 360$^{\circ}$ environment map. Third, the kitchen area is segmented from the indoor panorama, and new virtual kitchen components are positioned along the kitchen wall. Finally, the new kitchen space is rendered under global illumination, and the single panorama can be converted into 2D perspective views.
• Figure 3: Two Photometric Measurements: (a) A Konica Minolta LS-160 luminance meter (over $5,000 US dollars) is placed by the Ricoh Theta Z1 camera lens to measure the target luminance value on the whiteboard, as used in previous studies ji2023virtualji2024virtual. (b) Our low-cost approach: A TS-710 light meter (around $30 US dollars) is placed by the Ricoh Theta Z1 camera lens to measure the illuminance coming from 180$^{\circ}$ hemispherical directions.
• Figure 4: Using Illuminance Measurement to Calibrate HDR Panorama. (a) The content captured by the front lens is cropped and geometrically transformed into a 180$^{\circ}$ orthographic fisheye image. (b) Scene illuminance measured from the 180$^{\circ}$ hemispherical model is aligned to the fisheye image in orthographic projection.
• Figure 5: Comparison of the luminance values on the whiteboard using the standard approach (Konica Minolta LS-160 luminance meter) and our low-cost approach (TS-710 light meter) for photometric calibration.