T-Graph: Enhancing Sparse-view Camera Pose Estimation by Pairwise Translation Graph

TL;DR

T-Graph addresses sparse-view 6-DoF camera pose estimation by explicitly modeling pairwise translations among viewpoint pairs. It introduces a complete translation graph formed by a lightweight MLP-based regressor and provides two representations, relative-t and pair-t, to adapt to different camera configurations, all deployed as a training-time branch that does not affect inference. Across two strong baselines (RelPose++ and Forge-2D) and two public datasets (CO3D and IMC PhotoTourism), T-Graph yields consistent improvements in rotation, translation, and camera center accuracy, with pair-t excelling for center-facing, convergent setups and relative-t offering more stability for parallel-view configurations. The gains are achieved with minimal parameter overhead, underscoring the practical impact of leveraging inter-view translation information to enrich feature representations and global pose reasoning in sparse-view scenarios.

Abstract

Sparse-view camera pose estimation, which aims to estimate the 6-Degree-of-Freedom (6-DoF) poses from a limited number of images captured from different viewpoints, is a fundamental yet challenging problem in remote sensing applications. Existing methods often overlook the translation information between each pair of viewpoints, leading to suboptimal performance in sparse-view scenarios. To address this limitation, we introduce T-Graph, a lightweight, plug-and-play module to enhance camera pose estimation in sparse-view settings. T-graph takes paired image features as input and maps them through a Multilayer Perceptron (MLP). It then constructs a fully connected translation graph, where nodes represent cameras and edges encode their translation relationships. It can be seamlessly integrated into existing models as an additional branch in parallel with the original prediction, maintaining efficiency and ease of use. Furthermore, we introduce two pairwise translation representations, relative-t and pair-t, formulated under different local coordinate systems. While relative-t captures intuitive spatial relationships, pair-t offers a rotation-disentangled alternative. The two representations contribute to enhanced adaptability across diverse application scenarios, further improving our module's robustness. Extensive experiments on two state-of-the-art methods (RelPose++ and Forge) using public datasets (C03D and IMC PhotoTourism) validate both the effectiveness and generalizability of T-Graph. The results demonstrate consistent improvements across various metrics, notably camera center accuracy, which improves by 1% to 6% from 2 to 8 viewpoints.

Figures (8)

• Figure 1: Illustration of two pairwise translation representations based on different coordinate systems. The green cross indicates the origin of the world coordinate system. (a) Coordinate system of relative-t, (b) Coordinate system of pair-t.
• Figure 2: Comparison of camera pose estimation architectures with and without T-Graph. $\left\{ T_m \right\}_{m=1}^{n}$ represents the output (rotation, translation) of the baseline model and $\left\{ t_m \right\}_{m=1}^{C_n^2}$ represents the output (pairwise translation) of T-Graph. The pink region illustrates the baseline model structure without T-Graph, while the blue region shows the modified model structure after introducing T-Graph, which works as a parallel prediction branch to assist learning. Note that the T-Graph branch is only active during training for loss optimization and is removed during inference.
• Figure 3: T-Graph (simplified with four cameras) module with two different pairwise translation representations. (a) T-Graph, (b) Two types of pairwise translation representations.
• Figure 4: RelPose++ lin2024RelPose++ combined with T-Graph. RelPose++ adopts a ResNet-50 backbone to extract image features, which are then fused with positional embeddings and bounding box parameters and fed into a Transformer network. This method directly regresses camera translations while modeling the distribution of rotations through an energy-based approach.
• Figure 5: Forge-2D jiang2024few combined with T-Graph. Forge-2D employs a multi-layer CNN as a 2D encoder to extract image features. These features are then passed to a global pose feature extractor, which integrates cross-attention and self-attention mechanisms to capture pose-related information. Finally, an MLP jointly regresses camera rotations and translations in a unified manner.