Moving Light Adaptive Colonoscopy Reconstruction via Illumination-Attenuation-Aware 3D Gaussian Splatting

TL;DR

ColIAGS tackles illumination-driven photometric variation in colonoscopy 3D reconstruction by extending 3D Gaussian Splatting with two key modules: Improved Geometry Modeling with View Embedding (IMVE) and Improved Appearance Modeling with Illumination Attenuation (IMIA). It introduces cosine-based high-frequency embeddings and a joint illumination model that accounts for two attenuation factors, enabling accurate geometry alongside photorealistic rendering. The approach is trained with a depth-consistent loss and evaluated on C3VD and RotateColon, where it achieves a PSNR gain of 2.04 dB and a 78% reduction in Depth MSE relative to geometry-preserving baselines. These results demonstrate the practical potential for high-fidelity, real-time endoscopic reconstruction, with code to be released for community use.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a pivotal technique for real-time view synthesis in colonoscopy, enabling critical applications such as virtual colonoscopy and lesion tracking. However, the vanilla 3DGS assumes static illumination and that observed appearance depends solely on viewing angle, which causes incompatibility with the photometric variations in colonoscopic scenes induced by dynamic light source/camera. This mismatch forces most 3DGS methods to introduce structure-violating vaporous Gaussian blobs between the camera and tissues to compensate for illumination attenuation, ultimately degrading the quality of 3D reconstructions. Previous works only consider the illumination attenuation caused by light distance, ignoring the physical characters of light source and camera. In this paper, we propose ColIAGS, an improved 3DGS framework tailored for colonoscopy. To mimic realistic appearance under varying illumination, we introduce an Improved Appearance Modeling with two types of illumination attenuation factors, which enables Gaussians to adapt to photometric variations while preserving geometry accuracy. To ensure the geometry approximation condition of appearance modeling, we propose an Improved Geometry Modeling using high-dimensional view embedding to enhance Gaussian geometry attribute prediction. Furthermore, another cosine embedding input is leveraged to generate illumination attenuation solutions in an implicit manner. Comprehensive experimental results on standard benchmarks demonstrate that our proposed ColIAGS achieves the dual capabilities of novel view synthesis and accurate geometric reconstruction. It notably outperforms other state-of-the-art methods by achieving superior rendering fidelity while significantly reducing Depth MSE. Code will be available.

Figures (3)

• Figure 1: Overview of our proposed ColIAGS framework. The pipeline of ColIAGS contains two proposed components, i.e., the Improved Geometry Modeling with View Embedding(the yellow box) to enhance the geometry precision, ensuring the approximation conditions to incorporate illumination factors, and the Improved Appearance Modeling with Illumination Attenuation(the blue box) to model the two types of illumination attenuation.
• Figure 2: (a) illustrates the lighting attenuation in a conventional point light model, where the illumination depends on both the light source’s orientation and its distance to the object surface, while camera vignetting effects are influenced by the viewing direction. (b) presents a simplified lighting model, where the light source is approximated to be co-located with the camera, and its direction is assumed to align with the camera's orientation.
• Figure 3: Qualitative comparison on C3VD, against Scaffold-GS scaffold, Gaussians Pancakes pancakes and PR-Endo prendo.The top three rows display the rendered RGB images, while the bottom three rows show the depth mse error map.