Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AnimateScene: Camera-controllable Animation in Any Scene

Qingyang Liu, Bingjie Gao, Weiheng Huang, Jun Zhang, Zhongqian Sun, Yang Wei, Fengrui Liu, Zelin Peng, Qianli Ma, Shuai Yang, Zhaohe Liao, Haonan Zhao, Li Niu

TL;DR

This work proposes a training-free style alignment method that adapts the 4D human to match background illumination and appearance, achieving coherent composites and designs a joint post-reconstruction approach that incorporates camera trajectories for smooth, visually engaging motion videos.

Abstract

Recent advances in 3D scene reconstruction and 4D human animation have broadened adoption, but integrating the two remains difficult. Key challenges include placing humans at plausible locations and scales without interpenetration, aligning lighting and style between humans and background, and handling dynamic camera trajectories. We introduce AnimateScene, a unified framework addressing these issues. First, an accurate placement module automatically determines realistic 3D positions and prevents collisions during motion. Second, we propose a training-free style alignment method that adapts the 4D human to match background illumination and appearance, achieving coherent composites. Finally, we design a joint post-reconstruction approach that incorporates camera trajectories for smooth, visually engaging motion videos. Experiments across diverse scenes and actions demonstrate that AnimateScene generates dynamic results with high geometric detail and strong spatiotemporal coherence. The project page is available at https://whynothaha.github.io/AnimateScene/AnimateScene.html.

AnimateScene: Camera-controllable Animation in Any Scene

TL;DR

This work proposes a training-free style alignment method that adapts the 4D human to match background illumination and appearance, achieving coherent composites and designs a joint post-reconstruction approach that incorporates camera trajectories for smooth, visually engaging motion videos.

Abstract

Recent advances in 3D scene reconstruction and 4D human animation have broadened adoption, but integrating the two remains difficult. Key challenges include placing humans at plausible locations and scales without interpenetration, aligning lighting and style between humans and background, and handling dynamic camera trajectories. We introduce AnimateScene, a unified framework addressing these issues. First, an accurate placement module automatically determines realistic 3D positions and prevents collisions during motion. Second, we propose a training-free style alignment method that adapts the 4D human to match background illumination and appearance, achieving coherent composites. Finally, we design a joint post-reconstruction approach that incorporates camera trajectories for smooth, visually engaging motion videos. Experiments across diverse scenes and actions demonstrate that AnimateScene generates dynamic results with high geometric detail and strong spatiotemporal coherence. The project page is available at https://whynothaha.github.io/AnimateScene/AnimateScene.html.

Paper Structure

This paper contains 11 sections, 7 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Preliminaries
  4. Style Transfer Module
  5. Object Placement Module
  6. Joint Post‑Reconstruction
  7. Experiments
  8. Experimental setup
  9. Evaluation results
  10. Conclusion
  11. Acknowledgement

Figures (1)

  • Figure 1: Overview of AnimateScene. AnimateScene takes a single scene image, a single human image, an accompanying motion clip, and a user-defined camera path. It first aligns the human’s appearance with the scene. Next, it reconstructs a 4D Gaussian avatar alongside a sparse 3D Gaussian scene. Depth cues are then used to lift the avatar to a collision-free 3D location and scale. The system finally refines the fused human–scene field along the camera trajectory, inpainting any newly exposed regions. The result is a video where actor, environment, and viewpoint move in seamless geometric and stylistic harmony.