Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance

TL;DR

The paper addresses the lack of environment-aware fidelity in diffusion-based character animation by introducing Animate Anyone 2, which learns environment affordance from driving videos and populates environment regions with coherent characters.Key innovations include a shape-agnostic boundary mask for robust character-scene integration, an object guider with spatial blending to preserve interactive object dynamics, and a depth-wise pose modulation to support diverse motions.Experiments on diverse datasets demonstrate superior quantitative metrics and qualitative integration of characters with scenes and objects, surpassing prior methods and showing robustness to motion variety.The work enables more realistic character animation in complex environments with practical implications for filmmaking, advertising, and virtual character applications, while noting limitations such as hand-object interaction artifacts and reliance on segmentation tools.

Abstract

Recent character image animation methods based on diffusion models, such as Animate Anyone, have made significant progress in generating consistent and generalizable character animations. However, these approaches fail to produce reasonable associations between characters and their environments. To address this limitation, we introduce Animate Anyone 2, aiming to animate characters with environment affordance. Beyond extracting motion signals from source video, we additionally capture environmental representations as conditional inputs. The environment is formulated as the region with the exclusion of characters and our model generates characters to populate these regions while maintaining coherence with the environmental context. We propose a shape-agnostic mask strategy that more effectively characterizes the relationship between character and environment. Furthermore, to enhance the fidelity of object interactions, we leverage an object guider to extract features of interacting objects and employ spatial blending for feature injection. We also introduce a pose modulation strategy that enables the model to handle more diverse motion patterns. Experimental results demonstrate the superior performance of the proposed method.

Figures (9)

• Figure 1: We propose Animate Anyone 2, which differs from previous character image animation methods that solely utilize motion signals to animate characters. Our approach additionally extracts environmental representations from the driving video, thereby enabling character animation to exhibit environment affordance. The generated results demonstrate that, beyond maintaining character consistency, Animate Anyone 2 can produce high-fidelity results that seamlessly integrate characters with the surrounding environment.
• Figure 2: The framework of Animate Anyone 2. We capture environmental information from the source video. The environment is formulated as regions devoid of characters and incorporated as model input, enabling end-to-end learning of character-environment fusion. To preserve object interactions, we additionally inject features of objects interacting with the character. These object features are extracted by a lightweight object guider and merged into the denoising process via spatial blending. To handle more diverse motions, we propose a pose modulation approach to better represent the spatial relationships between body limbs.
• Figure 3: Different coefficients for mask formulation.
• Figure 4: Qualitative Results. Animate Anyone 2 achieves consistent character animation while enabling the integration and interaction between characters and their environments, thereby realizing environment affordance.
• Figure 5: Qualitative comparion for character animation. We normalize the background to a uniform color.