InfiniDepth: Arbitrary-Resolution and Fine-Grained Depth Estimation with Neural Implicit Fields

TL;DR

InfiniDepth reframes depth estimation as a neural implicit field to achieve arbitrary-resolution, fine-grained depth maps by querying depth at continuous image coordinates with a multi-scale local implicit decoder. A key advancement is the Infinite Depth Query, which allocates sub-pixel samples to produce nearly uniform 3D surface coverage, improving novel view synthesis under large viewpoint shifts. The approach is evaluated on a new Synth4K 4K-depth benchmark and real-world datasets, showing state-of-the-art performance on relative depth and competitive metric-depth results, with pronounced gains in high-frequency detail. The method enables high-fidelity depth-aware reconstruction and improved 3D perception pipelines, while acknowledging limitations in temporal consistency for videos and future work toward multi-view extension.

Abstract

Existing depth estimation methods are fundamentally limited to predicting depth on discrete image grids. Such representations restrict their scalability to arbitrary output resolutions and hinder the geometric detail recovery. This paper introduces InfiniDepth, which represents depth as neural implicit fields. Through a simple yet effective local implicit decoder, we can query depth at continuous 2D coordinates, enabling arbitrary-resolution and fine-grained depth estimation. To better assess our method's capabilities, we curate a high-quality 4K synthetic benchmark from five different games, spanning diverse scenes with rich geometric and appearance details. Extensive experiments demonstrate that InfiniDepth achieves state-of-the-art performance on both synthetic and real-world benchmarks across relative and metric depth estimation tasks, particularly excelling in fine-detail regions. It also benefits the task of novel view synthesis under large viewpoint shifts, producing high-quality results with fewer holes and artifacts.

Figures (13)

• Figure 1: InfiniDepth is a new depth representation which models depth as neural implicit fields, enabling arbitrary-resolution and fine-grained depth estimation. It also benefits novel view synthesis under large viewpoint shifts with fewer holes and artifacts.
• Figure 2: Overview of InfiniDepth. (a) Feature Query: given an input image and a continuous query 2D coordinate, we extract feature tokens from multiple layers of the ViT encoder, and query local features for the coordinate at each scale through bilinear interpolation. (b) Depth Decoding: given the multi-scale local features queried at the continuous coordinate, we hierarchically fuse features from high spatial resolution to low spatial resolution, and decode the fused feature to the depth value through a MLP head.
• Figure 3: Advantage of Our Infinite Depth Query. The blue and red highlighted regions represent areas with different depths, surface normals, and viewing directions. Per-pixel depth prediction leads to strong density imbalance in these regions due to perspective projection and surface orientation, while Infinite Depth Query applies sub-pixel query with adaptive weights to generate uniformly distributed point clouds.
• Figure 4: Normal map from implicit fields through torch autograd, indicating high-quality internal geometry of our model.
• Figure 5: Qualitative comparisons for relative depth estimation. The first two rows show prediction results on Synth4K, while the last row shows real-world data with low resolution RGB inputs. The boxes highlight detail regions upsampled to higher resolution for baselines, while our method directly predicts at this resolution. More comparisons can be found in the supp..