Z-SASLM: Zero-Shot Style-Aligned SLI Blending Latent Manipulation

TL;DR

Z-SASLM tackles the problem of achieving coherent multi-reference style alignment in diffusion-based text-to-image generation without fine-tuning. It introduces SLI Blending to interpolate style latents along the geodesic on the hypersphere, preserving latent-manifold structure and mitigating artifacts of linear mixing, with a weight-ordered aggregation for $k$ styles. A novel Weighted Multi-Style DINO ViT-B/8 metric is proposed to quantify cross-style coherence, complemented by an optional Multi-Modal Content Fusion pipeline that enriches conditioning with audio, image, and weather signals. Empirical results on Stable Diffusion XL show that SLI-based blending outperforms linear baselines in multi-style alignment and maintains robust performance under multi-modal prompts, while dynamic attention scaling and guidance analysis provide practical guidance for balancing style fidelity and content preservation. The work offers zero-shot, style-consistent generation across multiple references and modalities, with potential impact on creative workflows requiring cohesive multi-style imagery without costly fine-tuning.

Abstract

We introduce Z-SASLM, a Zero-Shot Style-Aligned SLI (Spherical Linear Interpolation) Blending Latent Manipulation pipeline that overcomes the limitations of current multi-style blending methods. Conventional approaches rely on linear blending, assuming a flat latent space leading to suboptimal results when integrating multiple reference styles. In contrast, our framework leverages the non-linear geometry of the latent space by using SLI Blending to combine weighted style representations. By interpolating along the geodesic on the hypersphere, Z-SASLM preserves the intrinsic structure of the latent space, ensuring high-fidelity and coherent blending of diverse styles - all without the need for fine-tuning. We further propose a new metric, Weighted Multi-Style DINO ViT-B/8, designed to quantitatively evaluate the consistency of the blended styles. While our primary focus is on the theoretical and practical advantages of SLI Blending for style manipulation, we also demonstrate its effectiveness in a multi-modal content fusion setting through comprehensive experimental studies. Experimental results show that Z-SASLM achieves enhanced and robust style alignment. The implementation code can be found at: https://github.com/alessioborgi/Z-SASLM.

Figures (7)

• Figure 1: Medieval-Cubism SLI Blending (2-styles)
• Figure 2: Overview of the Z-SASLM Architecture.
• Figure 3: Linear vs. SLI Blending with three styles. Linear interpolation can fall off the meaningful latent manifold, introducing artifacts. On the other hand, SLERP always provides robust results.
• Figure 4: 2-style Linear vs. SLI in the latent space Illustration. The chord (in red) represents naive linear blending between two reference vectors $\mathbf{z}_1$ and $\mathbf{z}_2$, yielding $\mathbf{z}_{\text{blend}}^{\text{lin}}$. In contrast, Spherical Linear Interpolation (in green) follows the geodesic on the unit sphere, producing $\mathbf{z}_{\text{blend}}^{\text{SLI}}$. The angle $\omega$ denotes the separation between $\mathbf{z}_1$ and $\mathbf{z}_2$, with $d_{\text{linear}} < d_{\text{sli}}$ on the unit circle.
• Figure 5: Z-SASLM with Multi-Modal Content Fusion. Our SLI blending (with $(w_{\text{med}}, w_{\text{vangogh}})$ = (0.15, 0.85)) preserves style alignment despite varied contextual cues (image, audio, music). As you can observe from the photos, the impact of the other modalities on the generated image is tangible (e.g., the background of the images is "heavy and thunderous" as the music modalities suggest). In the meanwhile, information given through Audio further allows to be more detailed in the generation.