RoboLight: A Dataset with Linearly Composable Illumination for Robotic Manipulation

TL;DR

This paper introduces RoboLight, the first real-world robotic manipulation dataset capturing synchronized episodes under systematically varied lighting conditions, and introduces RoboLight-Synthetic, comprising 196,000 episodes synthesized through interpolation in the HDR image space of RoboLight-Real.

Abstract

In this paper, we introduce RoboLight, the first real-world robotic manipulation dataset capturing synchronized episodes under systematically varied lighting conditions. RoboLight consists of two components. (a) RoboLight-Real contains 2,800 real-world episodes collected in our custom Light Cube setup, a calibrated system equipped with eight programmable RGB LED lights. It includes structured illumination variation along three independently controlled dimensions: color, direction, and intensity. Each dimension is paired with a dedicated task featuring objects of diverse geometries and materials to induce perceptual challenges. All image data are recorded in high-dynamic-range (HDR) format to preserve radiometric accuracy. Leveraging the linearity of light transport, we introduce (b) RoboLight-Synthetic, comprising 196,000 episodes synthesized through interpolation in the HDR image space of RoboLight-Real. In principle, RoboLight-Synthetic can be arbitrarily expanded by refining the interpolation granularity. We further verify the dataset quality through qualitative analysis and real-world policy roll-outs, analyzing task difficulty, distributional diversity, and the effectiveness of synthesized data. We additionally demonstrate three representative use cases of the proposed dataset. The full dataset, along with the system software and hardware design, will be released as open-source to support continued research.

Figures (9)

• Figure 1: The Light Cube system developed for controlled and repeatable robotic lighting data curation. We use this system to collect RoboLight, the first real-world robotic manipulation dataset capturing synchronized episodes under systematically varied lighting conditions.
• Figure 2: Overview of the HDR image processing pipeline. (a) Unprocessed RAW16 HDR image. (b-e) Effects of removing individual processing steps: bilateral denoising, lens shading correction, white balance correction, and color and gamma correction. (f) Final PNG output after full HDR processing, used for policy training.
• Figure 3:
• Figure 5: Examples from RoboLight-Synthetic. New episodes are synthesized by interpolating between HDR images from RoboLight-Real. In each row, only the first and last images are real; all intermediate images are synthesized.
• Figure 6: Initial object position distributions for the three tasks, each computed from 200 episodes. Coordinates are in millimeters and referenced to the robot base frame. Point density varies across tasks due to differences in object count. Within each task, object positions are randomized across placement patches.