Learnable Infinite Taylor Gaussian for Dynamic View Rendering

TL;DR

This work tackles dynamic novel view synthesis by modeling time-varying Gaussian properties within 3D Gaussian Splatting. It introduces a learnable infinite Taylor Formula where the dominant $3^{\text{rd}}$-order Taylor term governs motion and the Peano remainder is captured by a deformation field, achieving a robust, interpretable dynamic representation that blends implicit networks with explicit polynomial structure. Across N3DV and Technicolor datasets, the method attains state-of-the-art results, delivering higher fidelity and better temporal coherence than prior dynamic NeRF/GS approaches. The approach enables accurate, scalable dynamic scene reconstruction and high-quality novel-view rendering, with strong generalization to diverse scenes and motion patterns.

Abstract

Capturing the temporal evolution of Gaussian properties such as position, rotation, and scale is a challenging task due to the vast number of time-varying parameters and the limited photometric data available, which generally results in convergence issues, making it difficult to find an optimal solution. While feeding all inputs into an end-to-end neural network can effectively model complex temporal dynamics, this approach lacks explicit supervision and struggles to generate high-quality transformation fields. On the other hand, using time-conditioned polynomial functions to model Gaussian trajectories and orientations provides a more explicit and interpretable solution, but requires significant handcrafted effort and lacks generalizability across diverse scenes. To overcome these limitations, this paper introduces a novel approach based on a learnable infinite Taylor Formula to model the temporal evolution of Gaussians. This method offers both the flexibility of an implicit network-based approach and the interpretability of explicit polynomial functions, allowing for more robust and generalizable modeling of Gaussian dynamics across various dynamic scenes. Extensive experiments on dynamic novel view rendering tasks are conducted on public datasets, demonstrating that the proposed method achieves state-of-the-art performance in this domain. More information is available on our project page(https://ellisonking.github.io/TaylorGaussian).

Figures (6)

• Figure 1: The detailed architecture of the proposed method. The framework includes Gaussian Initialization, Sparse Point Sampling, Gaussian Point Interpolation, and Gaussian Transformation Fields Modeling.
• Figure 2: Comparison of novel view rendering on the N3DV dataset, with problem regions highlighted in boxes. More results can be found in the supplementary material and on our project website.
• Figure 3: Qualitative analysis of novel view rendering on the N3DV dataset, comparing the detail information of reconstructed images from different algorithms.
• Figure 4: Sear Steak Novel View Rendering on the N3DV Dataset: Qualitative Analysis of Ablation Experiments - Comparison of Reconstruction Quality and Detail Representation with Module Ablations.
• Figure 5: Qualitative analysis of novel view rendering on the Birthday dataset from the Technicolor, comparing the detailed reconstructions of different algorithms.