Dragging with Geometry: From Pixels to Geometry-Guided Image Editing

TL;DR

GeoDrag tackles the limitations of 2D-only point-based image editing by embedding 3D geometry into the editing process. It constructs a unified displacement field that blends a depth-aware geometry term $f_d$ with a plane-aware term $f_p$, fused as $f = (1 - rac{P}{P+oldsymbol{ extGamma}}) f_p + rac{P}{P+oldsymbol{ extGamma}} f_d$, and handles multi-point conflicts via conflict-free partitioning. Three core innovations—geometry-aware field modeling, spatial plane modulation, and region-based disjoint editing—enable fast, single-pass, structure-preserving edits with improved MD/DAI metrics and competitive perceptual fidelity. The method demonstrates strong practical impact for interactive editing tasks requiring precise geometry control, while maintaining efficiency suitable for real-time workflows. Overall, GeoDrag advances geometry-guided image manipulation by harmonizing 3D priors with 2D cues in a scalable, one-shot framework.

Abstract

Interactive point-based image editing serves as a controllable editor, enabling precise and flexible manipulation of image content. However, most drag-based methods operate primarily on the 2D pixel plane with limited use of 3D cues. As a result, they often produce imprecise and inconsistent edits, particularly in geometry-intensive scenarios such as rotations and perspective transformations. To address these limitations, we propose a novel geometry-guided drag-based image editing method - GeoDrag, which addresses three key challenges: 1) incorporating 3D geometric cues into pixel-level editing, 2) mitigating discontinuities caused by geometry-only guidance, and 3) resolving conflicts arising from multi-point dragging. Built upon a unified displacement field that jointly encodes 3D geometry and 2D spatial priors, GeoDrag enables coherent, high-fidelity, and structure-consistent editing in a single forward pass. In addition, a conflict-free partitioning strategy is introduced to isolate editing regions, effectively preventing interference and ensuring consistency. Extensive experiments across various editing scenarios validate the effectiveness of our method, showing superior precision, structural consistency, and reliable multi-point editability. The code will be available on https://github.com/xinyu-pu/GeoDrag .

Figures (20)

• Figure 1: (a) Displacement fields. Left: Plane-based estimation (e.g., FastDrag fastdrag) lacks geometric awareness and introduces structural inconsistencies under geometry-intensive edits like rotation. Right: Our geometry-aware GeoDrag aligns with 3D structure—near pixels move more, far pixels less. (b) Relying solely on 3D geometry can produce discontinuous displacements near object boundaries. (c) Two nearby handle-target pairs with opposing directions (e.g., leftward and rightward drags) may conflict, causing displacement cancellation and editing failure. The color legend (Shift/Distance/Depth) is provided in Appendix \ref{['appendix_df']}.
• Figure 2: Experimental Comparison. (a) Representative edits across diverse scenarios. (b) Quantitative results on DragBench: lower MD and DAI indicate more accurate editing. (c) Runtime and memory comparison across our GeoDrag and previous SoTAs.
• Figure 3: Overall framework of GeoDrag. In drag pipeline, the mask is split into sub-regions, each with a pair of drag points. For each sub-region, the geometry- and plane-aware displacement fields are independently calculated (see Sec. \ref{['gdf']} and Sec. \ref{['dff']}). Subsequently, these fused fields are aggregated without conflict (see Sec. \ref{['sec:agg']}). The final field enables one-step editing via latent relocation and interpolation, with reference guidance to preserve semantics.
• Figure 4: Illustration of $\boldsymbol{L}$ in the plane-aware field. $\boldsymbol{O}$ and $r$ are the center of the outer circle and radius, respectively.
• Figure 5: Qualitative comparisons with the state-of-the-art interactive point-based methods. See Appendix \ref{['more_visual_results']} for extended qualitative comparisons, including additional visualizations in Fig. \ref{['fig:appendix_comparison']}. In addition, see Appendix \ref{['effecr_of_masks']} for details on how mask selection influences editing behavior. Red points mark handles, and blue points mark targets; the same applies to the following figures.