Mono4DEditor: Text-Driven 4D Scene Editing from Monocular Video via Point-Level Localization of Language-Embedded Gaussians

TL;DR

Mono4DEditor addresses text-driven edits of 4D scenes reconstructed from monocular video by embedding language semantics into dynamic 3D Gaussians and localizing edits with a two-stage point-level approach. A language-embedded Gaussian representation enables semantic querying in 3D, while a diffusion-based video editor constrains changes to the localized region with optical-flow and scribble guidance to ensure temporal coherence. The method demonstrates precise region-specific edits that preserve unedited content and motion, outperforming previous 2D-diffusion–guided approaches in both visual fidelity and localization accuracy. This work offers a practical pathway for interactive, text-guided 4D content creation from casual monocular video, with implications for controllable editing in dynamic scenes.

Abstract

Editing 4D scenes reconstructed from monocular videos based on text prompts is a valuable yet challenging task with broad applications in content creation and virtual environments. The key difficulty lies in achieving semantically precise edits in localized regions of complex, dynamic scenes, while preserving the integrity of unedited content. To address this, we introduce Mono4DEditor, a novel framework for flexible and accurate text-driven 4D scene editing. Our method augments 3D Gaussians with quantized CLIP features to form a language-embedded dynamic representation, enabling efficient semantic querying of arbitrary spatial regions. We further propose a two-stage point-level localization strategy that first selects candidate Gaussians via CLIP similarity and then refines their spatial extent to improve accuracy. Finally, targeted edits are performed on localized regions using a diffusion-based video editing model, with flow and scribble guidance ensuring spatial fidelity and temporal coherence. Extensive experiments demonstrate that Mono4DEditor enables high-quality, text-driven edits across diverse scenes and object types, while preserving the appearance and geometry of unedited areas and surpassing prior approaches in both flexibility and visual fidelity.

Figures (9)

• Figure 1: Our approach Mono4DEditor allows users to edit 4D scenes from casual monocular video with text instruction. Mono4DEditor achieves precise, high-quality editing of the instructed content while maintaining irrelevant regions unchanged.
• Figure 2: Overview of our method. Given a monocular video , we construct a Language-Embedded Dynamic Gaussian field by enriching 3D Gaussians with quantized CLIP features (Section \ref{['sec:ledg']}). We then perform point-level localization to identify Gaussians relevant to the query, using 2D relevance maps and 3D semantic decoding (Section \ref{['sec:localization']}). Finally, we apply text-driven editing with a diffusion-based video model, modifying only the localized Gaussians to produce temporally consistent and spatially precise edits (Section \ref{['sec:editing']}). The colored visualization in the Language-Embedded Dynamic Gaussian field shows PCA results after semantic feature splatting.
• Figure 3: Comparison of editing results on the iPhone dataset. Our method achieves better temporal coherence, finer details (e.g., whiskers, eyes, specular highlights), and more accurate motion, while avoiding artifacts in unrelated regions. Baselines tend to over-edit or introduce distortions due to limitations in 2D diffusion-based approaches.
• Figure 4: Qualitative ablation study on the effect of localization refinement, including (1) input frame, (2) query mask based on the input prompt, (3) localized Gaussian rendering and edited result of the Full model, (4) result of w/o Ref, (5) result of w/o R-Ref and (6) result of w/o P-Ref.
• Figure 5: Editing results of Mono4DEditor on DAVIS and DyNerf datasets. Each example shows monocular video input and the text-driven edited output at two different time steps, rendered from two novel views. Our method achieves accurate localization, realistic appearance changes, and preserves spatial and temporal consistency across views.