Remotely Detectable Robot Policy Watermarking

TL;DR

This work tackles the problem of verifying the provenance of robotic policies using only remote observations, introducing the Physical Observation Gap as the core obstacle. It proposes Colored Noise Coherency (CoNoCo), a frequency-domain watermarking method that injects Colored Gaussian Noise into the policy's exploration signals and detects the watermark via spectral coherency, robust to unknown robot dynamics. The authors formalize the problem, prove marginal distribution preservation, and demonstrate robust remote detectability across simulated and real-world tasks and multiple observation modalities, including video, motion capture, and CCTV-like footage. This enables non-invasive IP protection and safety accountability for physical policies in robotics, with practical performance preserved and strong resistance to adversarial attempts.

Abstract

The success of machine learning for real-world robotic systems has created a new form of intellectual property: the trained policy. This raises a critical need for novel methods that verify ownership and detect unauthorized, possibly unsafe misuse. While watermarking is established in other domains, physical policies present a unique challenge: remote detection. Existing methods assume access to the robot's internal state, but auditors are often limited to external observations (e.g., video footage). This ``Physical Observation Gap'' means the watermark must be detected from signals that are noisy, asynchronous, and filtered by unknown system dynamics. We formalize this challenge using the concept of a \textit{glimpse sequence}, and introduce Colored Noise Coherency (CoNoCo), the first watermarking strategy designed for remote detection. CoNoCo embeds a spectral signal into the robot's motions by leveraging the policy's inherent stochasticity. To show it does not degrade performance, we prove CoNoCo preserves the marginal action distribution. Our experiments demonstrate strong, robust detection across various remote modalities, including motion capture and side-way/top-down video footage, in both simulated and real-world robot experiments. This work provides a necessary step toward protecting intellectual property in robotics, offering the first method for validating the provenance of physical policies non-invasively, using purely remote observations.

Figures (14)

• Figure 1: Overview of the pipeline for robot policy watermarking. In Step 1, the policy owner trains a policy, adds a watermark to it and produces a detection function to identify it. In Step 2, the watermarked policy is used by a policy user who deploys it on their own robot. In Step 3, a policy auditor aims to identify the policy used on the robot. To do so, they can only access glimpses of the policy behaviour through remote sensing, such as a camera feed; these glimpses are passed through the detection function to identify the policy.
• Figure 2: Overview of the Experimental Setup. (Left) Glimpse modalities: Ground Truth Action uses the watermarked action signal, Onboard Sensors uses readings from some onboard sensors; both assume the auditor can access some of the onboard hardware, Remote Motion Capture and Remote Camera Feed use only external sensors. (Right) Tasks: two are navigation tasks, either velocity- or force-controlled, the other two are actuated joints tasks, including an Inverted Pendulum and a Legged Robot, either force- or torque-controlled.
• Figure 3: Results on the RoboMaster Navigation tasks. (A) Example trajectories of the watermarked and non-watermarked policies on the robot. (B) Detectability: ROC curve for $40$ replications of the watermarked and non-watermarked policy for each baseline, lines indicate median and dashed areas quartiles. (C) Anonymity: computed as $1 -$ area under the ROC curve, for detection with a different seed. (D) Reward Preservation: reward distribution of the watermarked and non-watermarked policies.
• Figure 4: Results on a variety of Force and Torque Control tasks with increasing difficulty. (A) Detectability: ROC curve over $100$ replications of the watermarked and non-watermarked policy for each baseline, lines indicate median and dashed areas quartiles. (B) Anonymity: computed as the complement to $1$ of the ROC area under the curve for detection with a different owner seed, for Onboard Sensors glimpses. (C) Reward Preservation: reward distribution of the watermarked and non-watermarked policies.
• Figure 5: Relationship between glimpse sequence length and the watermark detectability of CoNoCo. Detectability is reported as the ROC AUC averaged over $10$ repetitions. We use the Onboard Sensors glimpse modality, except for “RoboMaster Navigation,” where we instead use real-world data from our robot experiments with Motion Capture glimpses. Shaded regions indicate quartiles.

Theorems & Definitions (9)

• Definition 3.1: Glimpse Sequence
• Definition 4.1: Complex Coherency
• Theorem 5.1
• Theorem 5.2: Invariance of Coherency Magnitude under LTI Filtering
• Definition 5.1: Signal-to-Interference-plus-Noise Ratio (SINR)
• Theorem 5.3: SINR in Watermarked Policies
• proof : Proof of Theorem \ref{['thm:W1']} (Marginal Distribution Preservation)
• proof
• proof