Chimera: Compositional Image Generation using Part-based Concepting

TL;DR

Chimera addresses the lack of explicit part-level control in personalized image generation by enabling assembly of novel objects from parts of multiple source images using only images and prompts, without masks. It introduces a large taxonomy of 464 semantic atoms and a synthetic 37k-prompt dataset to train a part-conditioned diffusion prior guided by part identity and spatial layout, via Grounded-SAM and IP+ embeddings feeding a finetuned PiT prior and SDXL decoder. The authors propose PartEval, an MLLM-based evaluator, and demonstrate that Chimera achieves superior part alignment and visual quality, especially as composition complexity grows, across animals and vehicles. The work enables practical applications in character creation, animation pipelines, and industrial design by integrating explicit part-level priors into generative workflows. Overall, Chimera shows that part-based conditioning is a scalable path toward controllable, high-fidelity multimodal generation.

Abstract

Personalized image generative models are highly proficient at synthesizing images from text or a single image, yet they lack explicit control for composing objects from specific parts of multiple source images without user specified masks or annotations. To address this, we introduce Chimera, a personalized image generation model that generates novel objects by combining specified parts from different source images according to textual instructions. To train our model, we first construct a dataset from a taxonomy built on 464 unique (part, subject) pairs, which we term semantic atoms. From this, we generate 37k prompts and synthesize the corresponding images with a high-fidelity text-to-image model. We train a custom diffusion prior model with part-conditional guidance, which steers the image-conditioning features to enforce both semantic identity and spatial layout. We also introduce an objective metric PartEval to assess the fidelity and compositional accuracy of generation pipelines. Human evaluations and our proposed metric show that Chimera outperforms other baselines by 14% in part alignment and compositional accuracy and 21% in visual quality.

Figures (12)

• Figure 1: Part-aware composition with Chimera: the model takes input images along with their specified part prompts (e.g., “head of a lion,” “body of a horse”) and generates a new entity that combines these parts into a coherent output.
• Figure 2: Scores assigned by human annotators for object generations of increasing complexity. For example, “2-part” refers to generating objects using two provided image-text pairs.
• Figure 3: Data creation pipeline: From a multi-domain part taxonomy, we sample 2-4 subject-part pairs and generate a prompt, render hybrids with HiDream-I1 Full, and collect images with ground-truth part masks for training and evaluation.
• Figure 4: An overview of our generation pipeline. User inputs are processed by Grounded-Segment-Anything (SAMv2) to produce segmented images, which are then encoded into the IP+ embedding space. This embedding conditions our finetuned PiT model, acting as a rectified flow prior, to generate a target latent that is subsequently rendered into the final image by the SDXL decoder.
• Figure 5: Our proposed MLLM-based evaluation pipeline. The process consists of three stages: (1) extracting structured features (e.g., color, texture) from a reference image, (2) automatically generating questions from these features, and (3) using Gemini-Flash to grade the generated image's faithfulness by answering the questions.