TrackGS: Optimizing COLMAP-Free 3D Gaussian Splatting with Global Track Constraints

TL;DR

TrackGS tackles the challenge of COLMAP-free novel view synthesis by bringing global feature tracks into 3D Gaussian Splatting. It introduces track Gaussians anchored to 3D tracks and two global track losses to enforce cross-view consistency, enabling end-to-end differentiable optimization of both intrinsics and extrinsics alongside the 3DGS parameters. The method demonstrates state-of-the-art pose accuracy and rendering quality on challenging real-world and synthetic datasets, outperforming prior COLMAP-free approaches that rely on local cues. By removing the need for COLMAP preprocessing and achieving robust global consistency, TrackGS broadens the practicality of 3DGS for real applications with diverse camera motions and unordered image collections.

Abstract

We present TrackGS, a novel method to integrate global feature tracks with 3D Gaussian Splatting (3DGS) for COLMAP-free novel view synthesis. While 3DGS delivers impressive rendering quality, its reliance on accurate precomputed camera parameters remains a significant limitation. Existing COLMAP-free approaches depend on local constraints that fail in complex scenarios. Our key innovation lies in leveraging feature tracks to establish global geometric constraints, enabling simultaneous optimization of camera parameters and 3D Gaussians. Specifically, we: (1) introduce track-constrained Gaussians that serve as geometric anchors, (2) propose novel 2D and 3D track losses to enforce multi-view consistency, and (3) derive differentiable formulations for camera intrinsics optimization. Extensive experiments on challenging real-world and synthetic datasets demonstrate state-of-the-art performance, with much lower pose error than previous methods while maintaining superior rendering quality. Our approach eliminates the need for COLMAP preprocessing, making 3DGS more accessible for practical applications.

Figures (6)

• Figure 1: Comparisons on novel view synthesis and camera poses. We propose a novel 3DGS model without any known camera parameters by leveraging global track information. Compared with state-of-the-art methods, we provide higher rendering quality in novel view synthesis, and more accurate estimation of camera poses on benchmark datasets, including the challenging real-world indoor and outdoor or synthetic scenes with complicated camera movements (the right column).
• Figure 2: Overview. Given a set of images, our method obtains both camera intrinsics and extrinsics, as well as a 3DGS model. During the initialization, we extract the global tracks, and initialize camera parameters and Gaussians from image correspondences and monodepth. We determine Gaussian kernels with 3D track points, and then jointly optimize the parameters $K$, $T_{cw}$, 3DGS through the proposed global track constraints ($L_{t2d}$, $L_{t3d}$) and original photometric losses ($L_1$, $L_{D-SSIM}$).
• Figure 3: Qualitative comparison for NVS on Tanks and Temples. We achieve better rendering results on details.
• Figure 4: Qualitative comparison for NVS and pose estimation on CO3D-V2. Benefit from the accuracy of the camera pose estimation, the rendering quality of novel view synthesis obtained by our method is higher than CF-3DGS.
• Figure 5: The trajectory of initial stage and joint stage. Our joint stage significantly improved the accuracy of camera pose.