BitMark: Watermarking Bitwise Autoregressive Image Generative Models

TL;DR

BitMark tackles the risk of model collapse from training on generated content by embedding a bit-level watermark directly into bitwise autoregressive image generators. It uses green/red bit lists and logit biasing to subtly steer bit sequences toward detectable patterns without harming image quality or speed, and proves radioactivity by propagating the watermark to downstream models. The approach shows strong robustness to a wide range of attacks, including dedicated watermark-removal strategies, and maintains fast detection. This work provides a practical, provenance-enabled defense to preserve model quality in large-scale image synthesis systems and extends to bitwise diffusion paradigms, with code available for reproducibility.

Abstract

State-of-the-art text-to-image models generate photorealistic images at an unprecedented speed. This work focuses on models that operate in a bitwise autoregressive manner over a discrete set of tokens that is practically infinite in size. However, their impressive generative power comes with a growing risk: as their outputs increasingly populate the Internet, they are likely to be scraped and reused as training data-potentially by the very same models. This phenomenon has been shown to lead to model collapse, where repeated training on generated content, especially from the models' own previous versions, causes a gradual degradation in performance. A promising mitigation strategy is watermarking, which embeds human-imperceptible yet detectable signals into generated images-enabling the identification of generated content. In this work, we introduce BitMark, a robust bitwise watermarking framework. Our method embeds a watermark directly at the bit level of the token stream during the image generation process. Our bitwise watermark subtly influences the bits to preserve visual fidelity and generation speed while remaining robust against a spectrum of removal techniques. Furthermore, it exhibits high radioactivity, i.e., when watermarked generated images are used to train another image generative model, this second model's outputs will also carry the watermark. The radioactive traces remain detectable even when only fine-tuning diffusion or image autoregressive models on images watermarked with our BitMark. Overall, our approach provides a principled step toward preventing model collapse in image generative models by enabling reliable detection of generated outputs. The code is available at https://github.com/sprintml/BitMark.

Figures (10)

• Figure 1: Bit and token overlaps in Infinity.
• Figure 2: Bit-Flipper attack for $\delta=2$
• Figure 3: Longer sequences are less consistent after re-encoding. We analyze the consistency of sequences of length $n$ after re-encoding over the whole image, following the setup of \ref{['fig:overlap_encoding_reencoding']}.
• Figure 4: BitMark is robust against rotation within $\pm 30$ degrees. The TPR@1%FPR for different $\delta$ and rotation degrees.
• Figure 5: Visualization of different images on different scales. The image on scale $i$ refers to the cumulative image obtained by adding up $1$ to $i$ scales. We visualize five sets of images: 1) Original images without watermarking, 2) Watermarked (last 3), where the watermarks are generated on scale 11 to 13, 3) Differences (last 3), which is the difference between the original and watermarked images (last 3), 4) Watermarked (all), where the watermarks are generated on all scales, 5) Differences (all), which is the difference between the original and watermarked images (all). We visualize scales 1-7.