Personalized Image Filter: Mastering Your Photographic Style

TL;DR

This work tackles learning and transferring professional photographic style from a few reference images while preserving the original content. It proposes a diffusion-prior based Personalized Image Filter (PIF) with a two-stage pipeline: first, a residual one-step diffusion backbone anchors the average photographic style; second, multi-concept inversion and textual inversion learn eight predefined photographic concepts via multiple pseudo words and a random-combination strategy. The method uses a content-preserving residual denoising mechanism and defines a photographic concept perturbation to modulate high-frequency details without content degradation, guided by losses that align region-specific features and attention maps. With quantitative and qualitative results on a new photographic dataset and standard content sources, PIF demonstrates superior concept transfer and content fidelity, while remaining interpretable and adaptable for interactive workflows in photomanipulation and personalization.

Abstract

Photographic style, as a composition of certain photographic concepts, is the charm behind renowned photographers. But learning and transferring photographic style need a profound understanding of how the photo is edited from the unknown original appearance. Previous works either fail to learn meaningful photographic concepts from reference images, or cannot preserve the content of the content image. To tackle these issues, we proposed a Personalized Image Filter (PIF). Based on a pretrained text-to-image diffusion model, the generative prior enables PIF to learn the average appearance of photographic concepts, as well as how to adjust them according to text prompts. PIF then learns the photographic style of reference images with the textual inversion technique, by optimizing the prompts for the photographic concepts. PIF shows outstanding performance in extracting and transferring various kinds of photographic style. Project page: https://pif.pages.dev/

Figures (14)

• Figure 1: Given several (a) reference images, PIF learns a set of (b) photographic (Photo.) concepts, and can render any (c) content image with the learned photographic style from (a), as shown in (d). In comparison, the text-based image editing method, SDEdit meng2021sdedit, and personalization method, DreamStyler ahn2024dreamstyler, change the content completely. The reference-based image retouching method PhotoWCT$^2$chiu2022photowct2 only transfers the color distribution instead of the photographic concepts.
• Figure 2: The eight types of photographic concepts employed in this paper, which are used to decompose the photographic style. The left and right halves of each image correspond to different appearances of the concepts, as described in the captions below.
• Figure 3: The pipeline of the residual one-step diffusion process, introduced in \ref{['sec:one_step']}. The photographic concepts in the input image $I$ are adjusted with the parameters of $\{\xi_j'\}$, while generating a description $y_\text{txt}$ of the target image $I'$. Given another adjusted image (parameterized by $\{\xi_j\}$), the noise prediction network is finetuned to predict the noise towards $I'$ by \ref{['eq:diffusion']}, to learn the average appearance and adjustment of the photographic concepts. To preserve the content, a residual denoising scheme is applied, anchoring the null text prompts $\varnothing$, as defined by \ref{['eq:res_denoise']}.
• Figure 4: The pipeline of photographic concept learning, described in \ref{['sec:PCLO']} where photographic concepts are randomly sampled from $P_\text{U}$ (illustrated in the top left bracket), and only the modifiers $[v_j]$ are optimized. The encoder and noise prediction model remain trained and fixed after \ref{['sec:one_step']}.
• Figure 5: A qualitative comparison of the proposed method and the baselines methods, where the proposed method manages to transfer the vignetting and high contrast from the reference image, and keeps the sushi its original appealing color. In contrast, most image retouching methods mistakenly modify the color towards unsaturated dark color, except for Deep Preset ho2021deep that introduces minor changes to the color.