Efficient 4D Gaussian Stream with Low Rank Adaptation

TL;DR

This work tackles dynamic novel view synthesis under continual learning by representing scenes with 3D Gaussian splats and applying a low-rank adaptation-based deformation model. The proposed LR-4DGStream fuses per-gaussian embeddings with a plane-based deformation encoder and a rank-$\lambda$ update to dramatically reduce streaming bandwidth while preserving competitive rendering quality. Evaluations on the DyNeRF dataset demonstrate 20 FPS rendering and bandwidth savings of about 90% compared with offline baselines, with PSNR/DSSIM/LPIPS comparable to state-of-the-art methods. The approach highlights the practicality of 4D Gaussian representations for real-time, long-sequence dynamic view synthesis, while noting challenges in cross-scene generalization for future work.

Abstract

Recent methods have made significant progress in synthesizing novel views with long video sequences. This paper proposes a highly scalable method for dynamic novel view synthesis with continual learning. We leverage the 3D Gaussians to represent the scene and a low-rank adaptation-based deformation model to capture the dynamic scene changes. Our method continuously reconstructs the dynamics with chunks of video frames, reduces the streaming bandwidth by $90\%$ while maintaining high rendering quality comparable to the off-line SOTA methods.