PixelShuffler: A Simple Image Translation Through Pixel Rearrangement
Omar Zamzam
TL;DR
This work targets efficient image-to-image translation, focusing on style transfer with minimal training overhead. It introduces PixelShuffler, a mutual-information-guided pixel shuffle that deforms the style image to align with the content while preserving colors, implemented via a deformation field predicted by a neural network. The key contributions are (a) a deformation-field framework that operates on the style image, (b) an mutual-information objective to balance content preservation with style integration, and (c) empirical results showing competitive LPIPS scores and solid FID performance without pre-training. The approach offers a lightweight, interpretable alternative to heavy diffusion or transformer-based style-transfer models and has potential applicability to broader image-to-image translation tasks, including medical imaging.
Abstract
Image-to-image translation is a topic in computer vision that has a vast range of use cases ranging from medical image translation, such as converting MRI scans to CT scans or to other MRI contrasts, to image colorization, super-resolution, domain adaptation, and generating photorealistic images from sketches or semantic maps. Image style transfer is also a widely researched application of image-to-image translation, where the goal is to synthesize an image that combines the content of one image with the style of another. Existing state-of-the-art methods often rely on complex neural networks, including diffusion models and language models, to achieve high-quality style transfer, but these methods can be computationally expensive and intricate to implement. In this paper, we propose a novel pixel shuffle method that addresses the image-to-image translation problem generally with a specific demonstrative application in style transfer. The proposed method approaches style transfer by shuffling the pixels of the style image such that the mutual information between the shuffled image and the content image is maximized. This approach inherently preserves the colors of the style image while ensuring that the structural details of the content image are retained in the stylized output. We demonstrate that this simple and straightforward method produces results that are comparable to state-of-the-art techniques, as measured by the Learned Perceptual Image Patch Similarity (LPIPS) loss for content preservation and the Fréchet Inception Distance (FID) score for style similarity. Our experiments validate that the proposed pixel shuffle method achieves competitive performance with significantly reduced complexity, offering a promising alternative for efficient image style transfer, as well as a promise in usability of the method in general image-to-image translation tasks.