On the Coexistence and Ensembling of Watermarks

TL;DR

This work shows that multiple deep image watermarks can coexist in the same image with only modest degradation, challenging assumptions that watermarks compete for the channel. It introduces the concept of a super watermark to identify detectors and demonstrates that coexisting watermarks can be ensembled post-training to increase total capacity without retraining. The study analyzes trade-offs among capacity, image quality, accuracy, and robustness, and introduces practical tools—strength clipping and error-correcting codes—to tune these trade-offs. The results suggest a pathway to scalable provenance in multi-actor ecosystems, but also open questions about the underlying non-linear channel dynamics and how best to exploit them in real-world applications.

Abstract

Watermarking, the practice of embedding imperceptible information into media such as images, videos, audio, and text, is essential for intellectual property protection, content provenance and attribution. The growing complexity of digital ecosystems necessitates watermarks for different uses to be embedded in the same media. However, to detect and decode all watermarks, they need to coexist well with one another. We perform the first study of coexistence of deep image watermarking methods and, contrary to intuition, we find that various open-source watermarks can coexist with only minor impacts on image quality and decoding robustness. The coexistence of watermarks also opens the avenue for ensembling watermarking methods. We show how ensembling can increase the overall message capacity and enable new trade-offs between capacity, accuracy, robustness and image quality, without needing to retrain the base models.

Figures (22)

• Figure 1: Two use-cases requiring coexistence of watermarks in the same image.A. A super watermark informs the user which watermark detector to use for this image. For a super watermark to work, it has to coexist with the main watermark. B. As different actors would use different watermarks for content provenance, intellectual property protection and content attribution, these all need to be able to coexist in the same image.
• Figure 2: Watermarks can coexist in the same image. We show accuracy, robustness and image quality when applying all possible pairs of methods. The left arrow for each metric indicates the change in performance of the first method with and without the second one; vice-versa for the right arrow. In general, two different watermarks can be added to the same image one after the other (in series) and both can be decoded with non-zero accuracy. However, coexistence tends to come at a small cost of accuracy, robustness and image quality. We have highlighted the cases where there is no significant reduction in accuracy and robustness.
• Figure 3: Controlling for the image quality degradation due to ensembling. We can reduce the image quality degradation in \ref{['fig:pairwise_plot_noclip']} due to ensembling by clipping the watermarks to strength 0.5 (see \ref{['sec:ensembles']} and \ref{['lst:clip_to_strength_code']} for details). Improving the image quality comes at a further drop in accuracy and robustness but some level of coexistence (i.e., non-zero accuracy for both methods, the endpoints of the leftmost pair of arrows in each cell) persists for most pairs of watermarks. We have highlighted the cases with accuracy $\geq$25%.
• Figure 4: Comparison of the tools for modifying existing watermarking methods. Combining strength clipping and ECCs with ensembling, as proposed, can lead to new trade-offs and, possibly, better models.
• Figure 5: Ensembling with another method can boost performance. These plots show two base watermarking methods (the starting points of the left and the right arrows for each pair) and their ensemble with strength clipping and ECC (the horizontal lines/arrow ends). In A. the ensemble has significantly larger capacity and image quality, in B. ensembling SSL with TrustMark B boosts its robustness and quality, in C. ensembling RoSteALS with SSL improves its accuracy and quality, and in D. ensembling HiDDeN with RoSteALS boosts its capacity, accuracy, robustness and quality. Numerical values can be found in \ref{['sec:extended_results']}.