DreamLoop: Controllable Cinemagraph Generation from a Single Photograph

TL;DR

DreamLoop addresses the challenge of generating cinemagraphs from a single photograph with flexible, user-guided control. It adapts a pretrained image-to-video diffusion model by introducing temporal-bridging to enable seamless looping and explicit motion conditioning via bounding-box sequences and sparse point tracks, leveraging $I_0$ and $I_T$ as conditioning signals and using $I_0=I_T=I$ at inference for loops. The approach achieves high-quality, controllable cinemagraphs across fluid-element and general-domain scenes, outperforming baselines on both objective metrics (e.g., $\text{FVD}$, $\text{KID}$, $\text{FID}$) and subjective user studies, while enabling fine-grained timing, full/partial motion paths, and region-specific control. This work significantly broadens access to cinemagraph creation by obviating the need for cinemagraph-specific training data and providing intuitive, image-centric controls with practical applications in creative media and AI-assisted videography.

Abstract

Cinemagraphs, which combine static photographs with selective, looping motion, offer unique artistic appeal. Generating them from a single photograph in a controllable manner is particularly challenging. Existing image-animation techniques are restricted to simple, low-frequency motions and operate only in narrow domains with repetitive textures like water and smoke. In contrast, large-scale video diffusion models are not tailored for cinemagraph constraints and lack the specialized data required to generate seamless, controlled loops. We present DreamLoop, a controllable video synthesis framework dedicated to generating cinemagraphs from a single photo without requiring any cinemagraph training data. Our key idea is to adapt a general video diffusion model by training it on two objectives: temporal bridging and motion conditioning. This strategy enables flexible cinemagraph generation. During inference, by using the input image as both the first- and last- frame condition, we enforce a seamless loop. By conditioning on static tracks, we maintain a static background. Finally, by providing a user-specified motion path for a target object, our method provides intuitive control over the animation's trajectory and timing. To our knowledge, DreamLoop is the first method to enable cinemagraph generation for general scenes with flexible and intuitive controls. We demonstrate that our method produces high-quality, complex cinemagraphs that align with user intent, outperforming existing approaches.

Figures (7)

• Figure 1: Teaser. Starting with the same input photograph, we showcase DreamLoop's capability to let user generate diverse cinemagraphs with precise motion control. (top-middle) the user intends to animate the earring of the girl, oscillating. (bottom-middle) the user simulates the action of 'girl petting the rabbit'. DreamLoop's generates high-quality cinemagraph following user input in both cases. User can also play with timing control, (left) user doubles the oscillating frequency of the girls earring and DreamLoop accurately captures it. We encourage readers to view the videos in the figure with Acrobat Reader.
• Figure 2: Methodology. Figure shows details of our method DreamLoop. (left) shows the training procedure with bounding box and sparse point track control. (right) shows the inference setting for generating controllable cinemagraphs with our method.
• Figure 3: Fine Grained Controls . (top) Timing controls: Highlights a scenario where timing control is essential for realistic motion. User intends to simulate the bead osciallting in simple harmonic motion (SHM). Without timing control, i.e., equally space points (top-left) the bead has constant motion, not following SHM, thus looking unrealistic. With timining control, user can assign more time the extrema and lesser time at the minima of SHM, accurately simulating physically realistic motion. (bottom) Full vs. Partial motion paths: Demonstrate 2 scenarios of the same example, where the user can either provide the full motion path (bottom-left) or just the initial motion trajectory (bottom-right) and remaining is automatically generated with our model.
• Figure 4: Comparison with baselines. Our full method (left) generates cinemagraphs with more realistic motion and accurately follows the input direction signal, both for the case of 'fluid-elements' cinemagraphs (top) and 'general-domain' cinemagraphs (bottom). (*) Denotes we apply postprocessing from endo2019animating to make the generated videos loop. (**) Denotes arrows on inputs for motion representative purpose. We encourage readers to view the videos in the figure with Acrobat Reader.
• Figure 5: Our Results. Figure shows the robustness of our method to generate very diverse cinemagraphs with different motion patterns, like cat moving its head, or human applying makeup in translation motion. The toy flower petals rotating. (bottom-left) Our method can also simulate complex, realistic hand-object interaction with relatively simple input control signals. We encourage readers to view the videos in the figure with Acrobat Reader.