Projection Mapping under Environmental Lighting by Replacing Room Lights with Heterogeneous Projectors

TL;DR

This work tackles the limitation of projection mapping in lit environments by substituting room lighting with a heterogeneous array of projectors that replicate ambient illumination around the PM target. It introduces a distributed projector optimization framework to reproduce environmental lighting and employs a large-aperture projector as an area light to illuminate surrounding surfaces, reducing hard shadows and high-luminance artifacts. Experimental results show improved PM contrast and realism under environmental lighting, and user studies indicate a shift from aperture-color to surface-color appearance, enhancing perceptual naturalness and collaborative usability. Together, the methods extend PM applicability to typical rooms, enabling more practical and communicative augmented surfaces without wearable devices.

Abstract

Projection mapping (PM) is a technique that enhances the appearance of real-world surfaces using projected images, enabling multiple people to view augmentations simultaneously, thereby facilitating communication and collaboration. However, PM typically requires a dark environment to achieve high-quality projections, limiting its practicality. In this paper, we overcome this limitation by replacing conventional room lighting with heterogeneous projectors. These projectors replicate environmental lighting by selectively illuminating the scene, excluding the projection target. Our contributions include a distributed projector optimization framework designed to effectively replicate environmental lighting and the incorporation of a large-aperture projector, in addition to standard projectors, to reduce high-luminance emitted rays and hard shadows -- undesirable factors for collaborative tasks in PM. We conducted a series of quantitative and qualitative experiments, including user studies, to validate our approach. Our findings demonstrate that our projector-based lighting system significantly enhances the contrast and realism of PM results even under environmental lighting compared to typical lights. Furthermore, our method facilitates a substantial shift in the perceived color mode from the undesirable aperture-color mode, where observers perceive the projected object as self-luminous, to the surface-color mode in PM.

Figures (11)

• Figure 1: Obtaining attenuation factor $p_1^{i_m}$ by projecting a uniformly white image from projector node 1, while other nodes project uniformly black image. The projected area of a physical projector, depicted by dashed lines, is divided into two segments. Projector nodes 1 and 2 represent virtual projectors, and their projected areas correspond to the respective segments.
• Figure 2: Aperture size of a projector affects the emitted ray intensity and the appearance of a shadow. (a) A standard projector is a point light source producing a high-luminance ray and a hard shadow. (b) The proposed large-aperture projector is an area light source creating a low-luminance ray and a soft shadow.
• Figure 3: An issue and its solution with the large-aperture projector illuminating the surrounding environmental surfaces of the PM target. (a) Some light rays intended to converge on an environmental surface point near the target unintentionally hit the target surface. (b) Turning off these pixels and compensating for the darkened area with the texture projectors.
• Figure 4: Experimental system. (a) A close-up view of our large-aperture projector. (b) An overview where standard projectors project uniformly colored images.
• Figure 5: The experimental results of the environmental lighting reproduction. (a) Projecting a uniformly white image from each projector node (indicated by the overlaid number) for obtaining $p_n^{i_m}$. (b) Captured images under three different types of environmental lighting; typical LED lights (target appearance), the reproduced lighting with projectors using a conventional method 1381255, and that using our method. (c) The appearance of the color chart under the three lighting conditions.