PICCOLO: Point Cloud-Centric Omnidirectional Localization

TL;DR

PICCOLO presents a training-free method for omnidirectional camera pose estimation by optimizing a point-cloud-centric sampling loss that compares colors sampled from a 360° panorama to a colored 3D point cloud. The core ideas are a differentiable sampling-based objective and a lightweight two-stage initialization that yields reliable starting points for gradient descent on SE(3). Empirical results across indoor and outdoor datasets, including the OmniScenes collection, show PICCOLO achieving high accuracy and stability, often outperforming existing omnidirectional localization methods. The approach leverages global scene context from the 360° view and demonstrates practical applicability with fast computation and minimal preprocessing, making it suitable for AR/VR and robotics scenarios.

Abstract

We present PICCOLO, a simple and efficient algorithm for omnidirectional localization. Given a colored point cloud and a 360 panorama image of a scene, our objective is to recover the camera pose at which the panorama image is taken. Our pipeline works in an off-the-shelf manner with a single image given as a query and does not require any training of neural networks or collecting ground-truth poses of images. Instead, we match each point cloud color to the holistic view of the panorama image with gradient-descent optimization to find the camera pose. Our loss function, called sampling loss, is point cloud-centric, evaluated at the projected location of every point in the point cloud. In contrast, conventional photometric loss is image-centric, comparing colors at each pixel location. With a simple change in the compared entities, sampling loss effectively overcomes the severe visual distortion of omnidirectional images, and enjoys the global context of the 360 view to handle challenging scenarios for visual localization. PICCOLO outperforms existing omnidirectional localization algorithms in both accuracy and stability when evaluated in various environments. Code is available at \url{https://github.com/82magnolia/panoramic-localization/}.

Figures (7)

• Figure 1: Overview of our approach. PICCOLO minimizes a novel, point cloud-centric loss function called sampling loss. After the initialization phase trims off local minima, PICCOLO minimizes the sampling loss with gradient descent.
• Figure 2: Qualitative results of PICCOLO. We display the input query image (top), and the projected point cloud under the estimated camera pose (bottom).
• Figure 3: Visualization of loss surfaces obtained from scenes in the Stanford2D-3D-S dataset stanford2d3d. The loss surfaces show the minimum loss values of the given ($x,y$) position in the 3D space. The red dots indicate the ground truth camera positions, and the blue dots link the values on the loss surface and the corresponding camera positions within the input point cloud space. Loss surfaces of small scenes are typically smooth with clear global minimum (left), but those of large scenes contain numerous local minima (right).
• Figure 4: Ablation study on the initialization pipeline.
• Figure 5: Additional qualitative results of PICCOLO with various input settings. We display the input query image (top) and the projected point cloud under the estimated camera pose (bottom).